SYSTEM AND METHOD FOR MONITORING USER 
INTERACTION WITH WEB PAGES 



FIELD OF THE INVENTION 

This invention relates to a system for monitoring usage of computers and other 
electronic devices, and, more particularly, to a system for collecting information concerning 
user interaction with application programs and the host environment in accordance with 
defined monitoring profiles. 

BACKGROUND OF THE INVENTION 

Many commercial and governmental organizations are becoming increasingly reliant 
upon software applications to perform enterprise management and other functions. 
Expenditures relating to hardware, software and related support (collectively, "information 
technology") now represent a significant portion of the capital improvements undertaken by 
many entities. Since it is not atypical for a large corporation to utilize software products from 
hundreds of different vendors, it is not uncommon for difficulties to arise because of 
overlapping program functionality. Usage of a multitude of software products also tends to 
complicate delineation of training and line management responsibilities. 

An enterprise may be motivated to make investments in information technology for a 
variety of reasons. For example, information technology assets may be utilized to generate 
revenue for the enterprise by facilitating accomplishment of essential tasks. Investments in 
information technology may also be motivated by a desire to reduce costs by streamlining or 
simplifying certain activities. Information technology may also be acquired as a form of 
insurance against losses from system failures or breaches in security, irrespective of whether 
the acquired assets enhance revenue or reduce costs. 

Although relatively clear motivations may exist for considering investments in 
information technology, the selection of particular assets for acquisition has become a 
complex process. A primary difficulty confronting managers responsible for acquiring 
information technology is determining an expected rate of return in connection with the 
proposed investment. This is a difficult task because current monitoring systems are not 
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designed to selectively track, measure and analyze user activity most pertinent to 
determination of such an expected rate of return. Responsible management must also decide 
which applications should be purchased, and when such purchases should be made. It must 
also be determined how users will be trained, and how subsequent performance will be 
measured. Current monitoring systems are also ill equipped to provide assistance in making 
these types of decisions. 

Management of large enterprises are also becoming interested in determining the total 
cost associated with ownership of particular information technology assets. A rudimentary 
measure of the cost of such ownership is obtained simply by determining the expense required 
to furnish each user with required hardware and associated software. More sophisticated 
measures of such ownership costs will also take into account an expected rate of return on the 
investment in information technology by ascertaining the extent of any increase in user 
productivity. Existing computer use monitoring systems have unfortunately not been 
specifically designed to measure such user productivity, and thus have tended to be of 
minimal assistance in facilitating determination of an expected rate of return on investments 
in information technology. Decisions regarding investments in information technology are 
thus likely being made without the benefit of relevant information relating to user productivity 
and expected rates of return. 

SUMMARY OF THE INVENTION 

In one aspect the present invention comprises a system for monitoring usage of an 
electronic device. A client component installed in a client device is operative to monitor 
usage of the client device in accordance with a monitoring profile, and to generate 
corresponding usage data. The monitoring profile typically includes information specifying 
which application programs, and which features of such application programs, installed on the 
client device are to be monitored by the client component. A server component, installed on a 
server device in communication with the client device, provides the monitoring profile to the 
client device and receives the usage data from the client device. The system may also include 
a data management component disposed to store the monitoring profile and to store the usage 
data provided to the server device. A data analysis component determines usage statistics 




5 associated with application programs installed on the client device based upon the usage data. 
The usage statistics may include measurements of usage time, number of uses, and sequence 
of usage of specified ones of the application programs. 

Another aspect the present invention relates to a method for monitoring user 
interaction with a web page downloaded to a client device from a remote location. 
10 Monitoring instrumentation is initially embedded within the web page in accordance with a 
monitoring profile. This embedding may be effected by, for example, incorporating scripting 
language into the web page. Events relating to such user interaction are monitored, and 
corresponding usage data is generated using the monitoring instrumentation. The usage data 
is then transmitted from the client device to a monitoring server. 

15 

BRIEF DESCRIPTION OF THE DRAWINGS 

ip* In the accompanying drawings: 

M3 FIG. 1 illustratively represents a distributed computing system in which the computer 

fg usage monitoring system of the present invention may be embodied. 

20 FIG. 2 is a block diagram of certain components contained in one of the client 

W computers depicted in FIG. 1. 

m 

" FIG. 3(a) provides a detailed representation of a client monitoring module operative to 

™ collect usage data on a client computer. 

p FIG 3(b) provides an illustrative overview of certain client-based and sever-based 

jij 25 components involved in monitoring user interaction with client computers in accordance with 
P the invention. 

FIG. 4 is an illustrative representation of an architecture for a profile and user 
management subsystem of the present invention. 

FIGS. 5 and 6 depict alternative types of associations between users and 
30 corresponding monitoring profiles. 
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FIG. 7 provides a representation of allowed associations between monitoring profiles 
on the one hand, and users, groups, and computers/subnets on the other hand. 

FIG. 8 depicts the relationship of a sever-based profile service to other components of 
a computer usage monitoring system of the present invention. 

FIG. 9 depicts a set of application profiles incorporated into a monitoring profile via 
association links. 

FIG. 10 illustratively represents potential associations between a set of application 
profiles and first and second monitoring profiles. 

FIG. 11 provides an illustrative representation of an architecture for tracking user 
interface events via Windows Hooks. 

FIG. 12 provides an illustrative representation of an architecture for tracking user 
interface events via Windows Hooks and events registered by Accessibility Objects. 

FIG. 13 illustratively represents the distributed architecture of a data analysis and 
display subsystem operative to collate and analyze the usage data cooperatively collected by 
client monitoring modules. 

FIG. 14 shows a data management subsystem disposed to store persistent objects used 
by other components of the inventive data monitoring system. 

FIG. 15 provides a representation of the architecture of an exemplary implementation 
of a system administration subsystem of the present invention. 

FIG. 16 is a flowchart representative of the process of restricting application feature 
usage in accordance with the invention. 

FIG. 17 depicts a view in the profile builder of an application profile containing 
restricted features. 

FIG. 18 illustratively represents a profile builder as included within a profile and user 
management subsystem. 

FIG. 19 depicts five main object types managed by the profile builder via its user 
interface. 

FIG. 20 is a class diagram representative of certain objects, and the relationships 
therebetween, managed by administrative users. 
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FIG. 21 shows a primary tree view from within the profile builder which enables users 



to navigate objects populating a system database. 

FIG. 22 illustratively represents the view afforded through selection of a "Group" tab 
of the user interface for the profile builder. 

FIG. 23 depicts the view available through selection of a "Monitoring Profile" tab of 
1 0 the user interface for the profile builder. 

FIG. 24 illustratively represents the contents of an "Application Profiles" view 
obtained through selection of an Application Profile tab of the user interface for the profile 
builder. 



1 5 manage profile data. 

FIG. 26 represents the relationships between various objects included within an 
exemplary implementation of a profile database. 



FIG. 25 depicts various interfaces of the database object broker used to access and 




execute on client machines. 



FIG. 27 illustratively represents an exemplary Monitoring Profile Database Schema. 
FIG. 28 illustratively represent the Monitoring Software components disposed to 



Ly FIG. 29 illustrates the interaction between a client service 50, a server module and a 



client monitoring agent. 



FIG. 30 depicts the organization of an IDataPacket. 



FIG. 3 1 represents a hierarchical network of server modules in accordance with one 



3 Js 25 aspect of the invention. 
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FIG. 32 illustrates the manner in which an Admin Console may serve as a user 
interface for a given server module. 

FIG. 33 illustratively represents the primary aspects of the relationship between a 
server module and a database object broker ("DBOB"). 

FIG. 34 illustrates the relationship between the profile service and various other 
system components. 

FIG. 35 provides an illustrative representation of the data report processing effected by 
the server module. 

FIG. 36 illustrates the reflexive client-server relationship between the server module 
and the client service. 

FIG. 37 is a state diagram representative of certain relationships between the client 
service and the server module. 

FIG. 38 illustratively represents the relationship between the server module and the 
client monitoring agent. 

FIG. 39 depicts the relationship between superior and subordinate server modules 
hierarchically arranged in accordance with one aspect of the invention. 

FIG. 40 provides an illustrative representation of the public interfaces exposed by the 
server module. 

FIG. 41 depicts the manner in which various report specifications are held within a 
system database. 

FIG. 42 highlights the client side components of a data management subsystem in 
accordance with the present invention. 

FIG. 43 highlights the server side components of the data management subsystem in 
accordance with the present invention. 

FIGS. 44 and 45 represent the process of accessing various database management 
systems from the DBOB via an ODBC interface. 

FIG. 46 represents certain aspects of the management of the DBOB via the Admin 
Console. 

FIG. 47 illustratively represents the client-server relationship between the server 
module and the DBOB. 



FIG. 48 shows one manner in which the profile builder is able to utilize the DBOB to 
manipulate profile database objects. 

FIG. 49 represents the simultaneous handling by the DBOB of multiple profile builder 

clients. 

FIG. 50 represents the manner in which DBOB is to construct and hold COM objects 
for data represented in a reports analysis database. 

FIG. 51 depicts a complete set of interfaces used between the DBOB and other system 

objects. 

FIG. 52 depicts the profile database objects and their interfaces. 
FIG. 53 depicts the report database objects and their interfaces. 

FIG. 54 highlights the client side components most directly associated with a system 
administration subsystem. 

FIG. 55 depicts the relationship between the Admin Console and a server module. 

FIG. 56 illustratively represents the propagation of administrative commands through 
a server tree in accordance with one aspect of the invention. 

FIG. 57 depicts the manner in which the Admin Console may ping a client service by 
making a call to CoGetClassObject() in order to obtain a Class ID (CLSID) maintained in the 
registry of such client during execution of the monitoring software of the present invention. 

FIG. 58 depicts the relationship between the Admin Console and the DBOB. 

FIG. 59 shows a view of the Admin Console user interface with a server tab selected. 

FIG. 60 illustrates certain information displayed by the Admin Console user interface 
upon selection of its server tab. 

FIG. 61 provides a screen shot of a data management dialog of the Admin Console 
user interface. 

FIG. 62 provides a screen shot of a "Client" tab of the Admin Console user interface. 
FIG. 63 provides a screen shot of a "Log" tab of the Admin Console user interface. 
FIG. 64 depicts the major system components of a web-based monitoring system of 
the present invention. 




FIGS. 65A and 65B illustratively represent two different patterns of a event handlers 
set up by a producer module 

FIG. 66 illustrates the objects involved in an instrumentation script and their 
interactions. 

FIG. 67 illustratively represents data transmission from a client browser to a remote 
server on the internet. 
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DETAILED DESCRIPTION OF THE INVENTION 
Overview 

A preferred embodiment of the computer usage monitoring system of the present 
invention may be used in a distributed system 10 of the type shown in FIG. 1 . The distributed 
system 10 includes a number of client computers 12 in communication with a server computer 
14 through a network 18. Each of the client computers 12 may concurrently run a client 
monitoring module 20 for tracking usage of application programs installed on the client 
computers 12 in accordance with monitoring profiles (described below). The network may be 
any of a number of different types of networks, including local area networks ("LANs") or 
wide area networks. Usage data collected by each client monitoring module 20 is provided to 
a server module 22 running on the server computer 14. The server module 22 also provides 
monitoring profiles to the client monitoring modules 20. 

FIG. 2 is a more detailed block diagram of the components contained in one of the 
client computers 12. Each of the client computers 12 need not have this configuration, and 
this configuration is intended to be merely illustrative. The client computer 12 includes a 
central processing unit ("CPU") 26 and a memory subsystem 28. The memory subsystem 28 
holds a copy of the operating system 27 for the client computer 12, such as the Microsoft 
Windows 95 Operating System sold by Microsoft Corporation. Also included within the 
memory subsystem 28 are RAM 29, application programs 30 and the client monitoring 
module 32, the latter two of which may be run on the CPU 26. 

The computer system 12 also includes a number of peripheral devices. These 
peripheral devices include a secondary storage device 40, such as a magnetic disk drive, a 
keyboard 42, a video display 44 and a mouse 46. 

FIGS. 3a and 3b provide more detailed representations of the client monitoring 
module 32 and associated server-based monitoring components. The client monitoring 
module 32 includes a client service 50 and a client monitoring agent 52, and is preferably 
implemented in the form of a Component Object Model ("COM") object operative within the 
Windows environment of the client computer 12. The Windows environment internally 
generates messages used by various modules to manage the operation of the computer and 



5 allocate its resources, and handles a vast array of overhead functions through the use of 
internal drivers. The internal drivers may include a Windows keyboard driver 56 and a 
Windows mouse driver 58. These drivers manage the overhead of manipulating mouse 
pointers, clicking the mouse buttons and entering information on the keyboard. User interface 
events such as mouse events and keyboard events are generated by the drivers 56 and 58. As 
10 is described below, each client monitoring module 32 monitors usage of application programs 
installed on its host client computer 12 in accordance with a monitoring profile. Each 
monitoring profile determines which application programs installed on a particular client 
computer 12 are to be monitored, and also specifies the usage data to be collected. 

In order to minimize complexity and facilitate orderly relations among the various 
15 monitoring components, each component is designed to have only limited awareness of the 
system environment. This limited awareness simplifies communication between system 
components, and eases system administration. In particular, each client monitoring agent 52 
is preferably made to be aware of only its client service 50. Similarly, each client service 50 
is aware of only its assigned server computer and of its client monitoring agents 52. As is 
^ 20 indicated by FIGS. 3a and 3b, the client service 50 is disposed to communicate with the server 
Ul module 22 20 on the server computer 14. This communication is preferably carried out in 

* accordance with the distributed COM protocol. The server module 22 20 accesses monitoring 

2 data and other information within the database 80 by way of a database object broker 

("DBOB") 74 (described below). 

25 

Monitoring and Application Profiles 
Referring now to FIG. 4, an illustrative representation is provided of an architecture 
for a profile and user management subsystem 70 of the present invention. The subsystem 70 
provides a mechanism through which monitoring profiles can be created, stored, and provided 
30 to client monitoring modules 32. The subsystem 70 includes a profile builder 72, and the 
DBOB 74 executing on the server computer 14. In a preferred configuration the profile 
builder 72 and DBOB 74 are implemented as separate COM objects, and communicate in 
accordance with the distributed COM protocol. 
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The profile builder 72 is an application program designed to provide a mechanism for 
creating a monitoring profile 76 composed of one more application profiles 78. Each 
application profile 78 determines the type of usage of an associated application program 79 
which is to be monitored. For example, an application profile 78 may specify that use of 
certain editing functions (e.g., "Cut" and "Paste" operations) are to be recorded, and that 
periods of inactivity of greater than a predetermined duration are also be noted. The 
monitoring profile 76 also specifies the frequency with which usage data collected in 
accordance with the application profiles 78 is to be reported to the server module 22 on the 
server computer 14. Such usage data may include, for example, information relating to the 
use of mouse actions verses accelerator keys, the frequency with which key application 
features are used, and the sequences of actions taken within an application or between 
applications. Each monitoring profile 76 may be assigned to a particular user or group of 
users. 

In a preferred implementation a set of default application profiles associated with well 
known application programs are stored within a database 80 for the server computer 14. 
Default monitoring profiles which use default application profiles may also be stored within 
the database 80. The default application profile provides a template from which monitoring 
profiles 76 and application profiles 78 tailored to particular users may be developed using the 
profile builder 72. 

One principal feature of the profile builder 72 is the ability to create application 
profiles applicable to essentially any Windows-based application. As is described below, the 
profile builder 72 allows for creation of an application profile 78 through selection of various 
items from the user interface presented by the associated application program. Items such as 
buttons, text fields, menus, and the like from the applicable user interface may be selected. 
Application profiles 78 that have been created in this manner via the profile builder 72 can be 
saved within the database 80 via DBOB 74 and used in combination with other monitoring 
profiles 78 in order to construct various monitoring profiles 76. The DBOB 74, in conjunction 
with the database 80, collectively serve as a repository for all monitoring profiles 76. The 
profile builder 72 uses the DBOB 74 to access an object corresponding to a monitoring profile 
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5 76 desired to be viewed and/or modified. The database 80 is preferably implemented as a 
relational database. 

The profile builder 72 will preferably provide a graphical interface allowing for 
inspection and modification of the monitoring profiles 76 stored within the database 80. In 
this regard each monitoring profile 76 can be visualized as a treelike structure consisting of a 

10 plurality of application profiles 78. Each constituent application profile 78 may be selected 
and its contents edited. As is described below, the profile builder 72 also allows for definition 
of particular classes of users, hereinafter referred to as "groups". A particular monitoring 
profile 76 may be assigned to each such group, thereby allowing evaluation of the 
performance of each user in the group relative to the assigned monitoring profile 76. Groups 

15 are a mechanism to allow system administrators to easily define and maintain identical 
profiles for multiple users. 
^ In a preferred implementation of the usage monitoring system of the present invention 

S B a monitoring profile 76 is associated with each user of a client computer 12. As is indicated 

if! 

ffl by FIGS. 5 and 6, this association may be in the form of a direct assignment of the monitoring 

i 

J: 20 profile 76 to a given user. Alternately, this association may be effected indirectly by virtue of 

W the user's inclusion in a group to which a monitoring profile 76 has been assigned. Each user 

On 

8 may be associated with only one group at a time, and each group associated with only one 

monitoring profile. By default, a user is assigned the monitoring profile associated with his 
Q group (FIG. 6). This can be overridden for individual users by assigning a monitoring profile 

25 directly to the user. In FIG. 6, the monitoring profile identified as "Profile 2" has been 
assigned directly to user Ul. If a monitoring profile 76 has not been so directly or indirectly 
assigned to a given user, the user's activity is monitored in accordance with a predefined 
default monitoring profile ("Default Monitoring Profile"). 

When a user becomes logged on to a client computer 12, the client service 50 installed 
30 on the computer 12 requests a profile server 88 to deliver the associated monitoring profile 
76. The client service then creates a client monitoring agent 52 and initializes it with the 
monitoring profile 76 (FIG. 8). In a preferred implementation each user is identified by a user 
name identical to the name used to identify the user within the applicable operating system 
(e.g., Windows 95 or Windows NT). Because it is possible for a previously unknown user to 

12 
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5 log onto a client computer 12, it is necessary to provide a mechanism for dealing with user 
names that are not otherwise recognized. Such an unknown user will often be encountered 
under the following circumstances: (i) a user logs onto a client computer 12 with a user name 
that has not been previously utilized; and (ii) a user may log onto a client computer 12 without 
specifying a user name, hereinafter an "anonymous user". In the former case, the profile 
10 builder 72 will add the new user to the list of known users and automatically assign the new 
user to a default user group (the "Default Group"). In the case described in (ii) above, the 
"Unknown User" becomes a member of the Default Group and is assigned the Default 
Monitoring Profile (FIG. 7). 

As an initial step in identifying the appropriate monitoring profile for a given user, 
15 upon becoming logged on to client computer 12 it is determined whether the user has 
previously been assigned a monitoring profile. If not, it is determined whether the user is a 
„ member of any defined groups. If so, the monitoring profile for the applicable group is 

C assigned to the user; otherwise, the profile for the default group is assigned to the user. Upon 

ill 

gj identifying the most specific monitoring profile applicable to the given user, the object 

2* 20 corresponding to the identified monitoring profile 76 is provided by the DBOB 74 to the 

iU client monitoring agent 52 (via the server module 22 20) by the client service 50. The client 

m 

monitoring agent 52 then begins monitoring user activity in accordance with the received 
y monitoring profile 76. 

p As is indicated by FIG. 9, one or more application profiles 78 are incorporated into a 

!i 25 monitoring profile 76 via association links. Each application profile 78 contains attribution 
P uniquely identifying those events to be recorded during the interaction of a user with a 

specific application program. It follows that each application profile 78 is specific to a 
particular application program, and in some cases is applicable only to a particular program 
version. 

30 Each monitoring profile will typically specify the application programs, and the 

features in such programs, which should be tracked. Monitoring profiles by default record 
entry into and exit from each application program in order to measure the duration of usage of 
each application. Various monitoring profiles may also define a user inactivity threshold, 
which is compared to the duration of a user's inactivity. A user is deemed to be "inactive" 
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5 when such user fails to interact with any application program for a period exceeding the 
inactivity threshold, and is otherwise deemed "active". Information relating to the schedule 
(e.g., time of day, days of week) during which a given user is to be monitored will be included 
in each monitoring profile 76. 



10 reporting of monitoring data from the client service 50 to the server module 22. These data 
reporting specifications will typically specify an upper limit to the size of the file (or memory 
buffer) containing the monitoring data held on the client computer 12. In a preferred 
implementation the client monitoring agent 52 informs the client service 50 when this upper 
limit is exceeded (i.e., when data is available for transfer from the client monitoring agent 52). 
15 In turn, the client service 50 signals the applicable server module 22, which in its discretion 
calls back to the signaling client service 50 to collect the cached data. In a preferred 
□ implementation a nominal data reporting interval, relating to the minimum frequency with 



which reports need to be provided to the server module 22, is included in the data reporting 
specifications of each monitoring profile. This parameter ensures that the server module 22 



p I 20 obtains timely updates from the client monitoring agent 52 (i.e., at least once per nominal data 



reporting interval), even during periods of low user activity. 

FIG. 10 illustratively represents potential associations between a set of application 
profiles 78a-78d and first and second monitoring profiles 76a and 76b. In order to facilitate 



advantageously obviates the need to modify each monitoring profile with which a given 
application profile is associated upon making changes to such application profile. FIG. 10 
should not be interpreted as implying that a particular application program may only be 
characterized by a single application profile. Rather, FIG. 10 illustratively represents that 
30 multiple application profiles may exist for a single application program even in the context of 
a single monitoring profile. In such a case a composite application profile for the application 
program could be generated by merging the specific application profiles. This merging 
process would involve concatenating the different program features to be tracked from each 



Also preferably included in each monitoring profile are specifications pertaining to the 
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constituent application profile such that usage of each program feature is only reported once 

during each reporting period. 

Each client monitoring agent 52 is also capable of collecting, in accordance with the 

applicable application profile, data pertaining to usage of defined "features" of application 

programs. In the context of the present invention the term "feature" is intended to refer to the 

operation or function performed by a series of keystrokes or mouse clicks. For example, the 

keystrokes required to enter text into a file path field, or to enter text into a main application 

window, could each be defined to correspond to a particular feature. 

In an exemplary implementation the client monitoring agent 52 is disposed to track the 

following application features: 

Menu Item Usage (via mouse and accelerator keys). Usage of menu items, whether 
invoked by mouse or keyboard actions. 

Toolbar Control Usage. Usage of commonly seen "bitmap buttons" on toolbars, as well 

as utilization of more complex controls embedded on toolbars. 

Dialog Usage. When, and how many times, a particular dialog is used. 

Top Level Window Access, Requires determination of when a given top level window of 

an application having multiple top level windows has been accessed. 

Control Usage. Use of individual controls appearing on various windows, including 

control usage within specific dialogs and popup windows. 

Main Application Window State. Refers to the state of the main application window 

during use of the subject application program (e.g., whether the window was maximized, 

the window's position on the display screen, movement of the window). 

Popup Menu Usage. Use of popup menus and specific menu items. 

Duration of Feature Use. The time interval during which a user interacts with a given 

control (e.g., a text field). 

Control State. The state of a given control upon activation by a user. As an example, the 
text typed into a text box (typically after truncation) could correspond to a Control State, 
as could the state of check boxes in a given application. 
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Methods of Tracking Application Usage 
As is described in detail below, the client monitoring agent 52 generally uses one or 
both of two primary methods of collecting information relating to usage of application 
programs. A first technique involves monitoring and filtering the message traffic associated 
with particular user interface events of Windows-based applications. This information is 
accessed via hooks in the applications and Windows operating system which are associated 
with particular user interface events (collectively, "Windows Hooks"). Data relating to this 
message traffic may be analyzed to ascertain the existence of activities deemed to be of 
significance within an application. Such activities could include, for example, rates of mouse 
and accelerator key events useful in computation of general productivity metrics. 

FIG. 11 provides an illustrative representation of an architecture for tracking user 
interface events via Windows Hooks. As is indicated by FIG. 11, a dynamic linked library 
("Hooks DLL") 92 is injected by the client monitoring agent 52 into each application program 
94 that has been activated by the user. A Hooks DLL 92d generated by the client monitoring 
agent 52 is also used to monitor user interaction with the desktop environment 96. The Hooks 
DLLs are inclined to monitor prescribed Windows Hooks in the application programs 94 and 
desktop environment 96 so as to determine when specific user input events (e.g., mouse clicks 
and keystrokes) have been performed. Information relating to the monitored user actions is 
reported by the Hooks DLLs to the client monitoring agent 52, which then forwards the 
collected information to the client service 50 as described above. The client monitoring agent 
52 and the application program hosting the Hooks DLLs 92 are preferably implemented as 
distinct processes with their own memory and processing threads. The Hooks DLLs 92 utilize 
the processing threads of the host application program being monitored. In the 
implementation of FIG. 21 the injected Hooks DLLs 92 and the client monitoring agent 52 
communicate via asynchronous shared memory. 

In implementations of the present invention within systems governed by recent 
versions of the Microsoft Windows operating systems (e.g., Windows 95, Windows 98, or 
Windows NT 5.0), information concerning usage of certain application programs may also be 
collected via Microsoft Active Accessibility application programming interfaces 
("Accessibility APIs"). These Accessibility APIs allow more extensive information to be 
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collected regarding usage of various application features. In this context an application 
"feature" is defined by an Accessible Object of the application program, where the term 
"Accessible Object" is intended to have the meaning provided within the "Microsoft Active 
Accessibility Software Development Kit," which has been made available at 
http://www.microsoft.com/enable/dev/msaa.htm . Accessibility APIs were designed primarily 
to provide limited internal access to an application program in order to facilitate utilization of 
alternative input and feedback mechanisms associated with third party applications. 

Application programs incorporating Accessibility APIs create and destroy Accessible 
Objects during program execution. In a preferred implementation of the present invention the 
following process is employed to track usage of application features by monitoring associated 
Accessible Objects: 

1 . An administrator identifies the features to be tracked in a set of chosen applications. 

2. This information is stored in an application profile, which is incorporated as described 
above into the monitoring profile associated with particular user(s). 

3. Use of specific features within each chosen application are tracked according to the 
application profile during usage of the chosen applications by such user(s). 

Since Accessible Objects are extinguished each time an application program is terminated, it 
is necessary to identify those of such objects corresponding to monitored features upon each 
subsequent invocation of the application program. TABLE I provides a list of attributes of 
Accessible Objects and accompanying explanations of the manner in which such attributes 
can assist in identifying objects corresponding to monitored features. 
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Table I 



Attribute Explanation 



Object Name 



Role 



Window 
Class 



Top Level 

Window 

Class 



The Object Name is the name as reported by the object's accessibility 
interface. The Object name is usually (but not always) the same as its 
displayed caption. For example, a button's name is typically the same as its 
label. The same is true for menu items. 

The Role is the Accessible Object's Role as reported by the objects 
accessibility interface. The set of allowable roles is defined by Microsoft 
as part of the Accessibly API. There are presently 61 defined roles. The 
role of an object is used to help identify when a particular feature is used. 
Roles include denotations such as button, menu item, popup menu, combo 
box and others. 

The Window class is the class name of the Object's parent window. The 
window class is included in the identification attributes to help distinguish 
between objects that have the same name and role, but whose window 
class may be different. 

This is the Top Level Window's Class of the window at the top of the 
Microsoft Window's window hierarchy chain. 



The client monitoring agent 52 examines the attributes of Accessible Objects set forth 
in Table I to identify objects corresponding to monitored features. However, examination of 
the four attributes listed in Table I will not always uniquely identify a monitored, although in 
most practical applications such an examination will suffice. As an example, consider the case 
in which the monitored feature consists of the Copy menu item on the Edit menu within 
Microsoft Word 97. The Copy menu item reports the following characteristics when 
examined through its accessible interface: 

• The Object Name is "Copy." 

• The Role is ROLE_SYSTEM_MENUITEM 

• The Window Class is "MsoCommandBar" 

• The Top Level Window Class is "Opus App" 
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5 Unfortunately, these features are also reported by the Accessible Object associated with a 
Copy menu item appearing on the popup menu which arises in response to a "right click" in 
the text area of a document. Accordingly, merely examining the values for the attributes 
listed in Table I leads to an ambiguous result. Table II lists the values for the attributes of 
the Accessibility Object for the "Copy" feature from the popup menu along with the attributes 
10 associated with the "Copy" feature from the menu bar: 

Table II 



hi 

ffl 
p 

o 



Attribute 


(Copy) Popup Menu 


(Copy) Edit Menu on Menu Bar 


Object Name 


Copy 


Copy 


Object Role 


ROLESYSTEMMENUITEM 


ROLESYSTEMMENUITEM 


Window Class 


MsoCommandBar 


MsoCommandBarPopup 



Top Level Window OpusApp MsoCommandBarPopup 
Class 



15 In the example of TABLE II, the values of the Window Class and Top Level Window 

jrj differentiate between the two Copy items. Although it is believed that in most instances 

^! Accessibility Objects associated with specific features may be distinguished by inspection of 

fy the attributes described in Table I and Table II, the present invention provides a mechanism 

hereinafter referred to as "feature ancestry" for resolving any ambiguities. 

20 The concept of feature ancestry may be appreciated by considering that every 

Accessible Object is a member of a tree structure of Accessible Objects stemming from a Top 

Level Window. In this context a given Accessible Object can generally be identified by a 

qualified name generated by concatenating the names of selected Accessibility Objects 

forming an ancestry of the given Accessible Object. It is noted that each potential ancestral 

25 Accessibility Object need not necessarily be used in the identification process. The following 

procedure may be utilized to determine the qualified name of an Accessible Object: 

1 . For each ancestor, specify whether or not such ancestor is to be used for identification 
purposes. 

30 2. For each ancestor being used for identification purposes, select one or any 
combination of the following three attributes for examination: Object Name, Role, and 
Window Class. 

As an example of the above procedure, consider the case of the standard "OK" button 

35 commonly found on dialog boxes. In the case of Microsoft Word 97, examination of the 
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attributes of the Accessibility Object for the "OK" button on the dialog box brought up via 
the "Tools/Options..." menu item yields the following: Object Name is "OK", Role is 
ROLESYSTEMPUSHBUTTON, Window Class is "bosa_sdm_Microsoft Word 8.0", and 
Top Level Window Class is "bosa_sdm_Microsoft Word 8.0". Examination of the 
Accessibility Object for the "OK" button on the dialog box brought up by "File/Print..." 
menu item yields exactly the same information. However, the ancestral Accessibility Objects 
for each of these "OK" buttons are different as indicated by Table III: 

Table III 







Tools/Options... Dialog 


File/Print... Dialog 






OK button 


OK button 


Ancestor! 


Name: 


Options 


Print 




Role: 


ROLESYSTEMDIALOG 


ROLESYSTEMDIALOG 




Class: 


bosasdmMicrosoft Word 8.0 


bosa sdm Microsoft Word 8.0 


Ancestor2 


Name: 


Options 


Print 




Role: 


ROLE_SYSTEM_WINDOW 


ROLE_SYSTEM_WINDOW 




Class: 


bosa_sdm_Microsoft Word 8.0 


bosa sdm Microsoft Word 8.0 



As is indicated by Table III, the Accessibility Objects for the two "OK" buttons described in 
the preceding example can be distinguished only by comparing their respective ancestral 
Accessibility Objects. 

Each feature identified by the applicable monitoring profile is tracked according to 
whether the feature is "used" or not. When features are monitored via associated 
Accessibility Objects, the definition of usage for a given feature is dependent upon the nature 
of the role defined by its associated Accessibility Object. Table IV below provides 
exemplary usage definitions for roles assumed by various Accessibility Objects associated 
with features likely to be tracked. Again, the role of an Accessibility Object may be 
examined by way of the defined set of Accessibility APIs. Table IV also provides, for 
certain roles, examples of the manner in which usage of an application program affects the 
state of a given role. 

Table IV 

ROLE_SYSTEM_BUTTONDROPDOWN 

1 . Used if pressed. 

No Application Instances Appear To Exist. 

role_system_buttondropdowngrid 
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1. Used if pressed. 

No Application Instances Appear To Exist. 
ROLE_SYSTEM_BUTTONMENU 

1 . Used when Pressed. 

No Application Instances Appear To Exist 
ROLE_SYSTEM__CHECKBUTTON 

1 . Used when State Changed (i.e. transitioned from check to unchecked or vice-versa) 

Check buttons are the commonly found check boxes in lots of places. The standard windows check box 

control has this role. 
ROLEjSYSTEM_CLIENT 

1 . Used when receives mouse clicks or keystrokes. 

Clients are frequently found as the interior area of framed windows. The interior area of Word 
documents are clients, as are excel and all tested applications. 

An object that is a Client is defined as Used if any mouse clicks or keystrokes are aimed at it. 
ROLE_SYSTEM_COMBOBOX 

1 . Used when any of its children are used. 

A combo box always has three children: the text area (sometime editable, sometimes not) the button the 

drops and undrops the list (sometimes visible, sometimes not), the window that holds the list 

A combo box is defined as used if the user uses and of the child windows. 

The standard windows combo box has this role. 
ROLE_SYSTEM_DIAL 

1 . Used if child objects are used or if the dial object receives any mouse clicks or keystrokes. 

An object that is a dial is used if any of its child objects are used or if the object itself receives any 

mouse clicks or keystrokes. 

No Application Instances Appear To Exist 
ROLESYSTEMDIALOG 

1 . Used when displayed to the user. 
ROLE_SYSTEM_DROPLIST 

1 . Used if any of its child objects are used. 

A drop list is like a combo box, it always has three children: the text area (sometime editable, 
sometimes not) the button the drops and undrops the list (sometimes visible, sometimes not), the 
window that holds the list 

A list box is defined as used if the user uses any of the child objects. 

No Application Instances Appear To Exist. 
ROLE_SYSTEM_GRIP 

1 . Used if it receives any mouse down messages. 

No Application instances Appear To Exist 
ROLE_SYSTEM_HOTKEYFIELD 

1 . Used if receives any keystrokes. 

Used if it receives any keystrokes, 
ROLE_SYSTEM_LINK 

1. Pressed. 

Links act like buttons, but generally look like text or graphics. 
ROLE_SYSTEM_LIST 

1 . Used when one or multiple list items are selected/deselected. 
ROLE_SYSTEM_LISTITEM 

1 . A list item is used whenever its state toggles between selected and not selected. 
ROLE_SYSTEMJVIENUITEM 

1 . Used when the menu item is selected. 

A menu item is selected when its state becomes "focused" and while focused it receives a mouse up 

event or return key press. 
ROLE_SYSTEMJVIENUPOPUP 

1 . Used if any of its children are used. 
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ROLE_SYSTEM_OUTLINE 

1 . Used if any mouse clicks or keystrokes are directed at it or if any of its children are used. 

The Outline is used if any mouse clicks or keystrokes are directed at it or if any of its children are used. 

The standard windows tree control publishes its role as outline. 
ROLE_SYSTEM_OUTLINEITEM 

1 . Outline Items are used if they receive any keystrokes or mouse clicks or the object's state toggles 
between selected / unselected. 

Items within the standard windows tree control publish their role as outline. 
ROLE_SYSTEM_PAGETAB 

1 . Used if any mouse clicks are directed at it. 

Tabs on tabbed dialogs appear as page tabs. 
ROLE_SYSTEM_PROPERTYPAGE 

1 . Used if any child objects are used. 

Used means the page was displayed. 
ROLE_SYSTEM_PUSHBUTTON 

1 . Used if pressed. 

The button was pressed, either by the user interacting with the control via the mouse or keyboard. 
ROLE_SYSTEM_RADIOBUTTON 

1 . Used if the button transitions between checked an unchecked states. 

A radio button is used if it transitions between the checked and unchecked states. 
ROLE_SYSTEM_SCROLLBAR 

1 . Used if any of its child objects are used or if the object itself receives any keystroke or mouse 
clicks. 

A scroll bar is used if any of its children are used. 
ROLESYSTEM_SLIDER 

1 . Used if the object receives any mouse clicks or keystrokes. 

The standard windows slider control publishes this role. 
ROLE_SYSTEM_SPINBUTTON 

1 . Used if the user clicks the mouse within the spin object 
ROLE_SYSTEM_TEXT 

1. Used if there are any keystrokes or mouse clicks directed at the object. 
ROLESYSTEMTITLEBAR 

1. Used (gabbed) if a mouse down occurs on the object. 
ROLE_SYSTEM_TOOLBAR 

1 . Used if any of its children are used. 
ROLE_SYSTEM_WINDOW 

1 . Used if any of its children are used. 

FIG. 12 provides an illustrative representation of an architecture for tracking user 
interface events via Windows Hooks and events registered by Accessibility Objects 
("Accessible Events"). As is indicated by FIG. 12, a Hooks DLL 92 and an application 
features DLL ("AppFeature DLL") 102 is injected by the client monitoring agent 52 into each 
application program 94 that has been activated by the user. A Hooks DLL 92d and an 
AppFeature DLL 1 02d generated by the client monitoring agent 52 are also used to monitor 
user interaction with the desktop environment 96. The Hooks DLLs 92 monitor prescribed 
hooks in the application programs 94 and desktop environment 96 so as to determine when 
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5 specific user input events (e.g., mouse clicks and keystrokes) have been performed. The 
AppFeature DLLs 102 are designed to monitor Accessible Events in order to determine 
whether each such Accessible Event relates to an Accessible Object for a feature being 
tracked. 

In accordance with one aspect of the present invention, the AppFeature DLLs 102 
10 internally use Windows Hooks in conjunction with Accessible Events to determine whether a 
feature is being used. Although the Accessibility API provides notification of occurrences of 
Accessible Events monitored by AppFeature DLLs 102, not all Accessible Objects generate 
notifications capable of being monitored by the AppFeature DLLs 102. Accordingly, the 
Windows-based events tracked by the Hooks DLLs 92 are used to trigger the AppFeature 
15 DLLs 102 to query the Accessibility API for information relating to the identity of the 
Accessible Object, if any, associated with the Window generating the event. The following is 
an exemplary sequence of Windows-based events capable of being monitored by an 
AppFeature DLL 102: 

WM KEYDOWN (Win32 Event) 

20 WM_CHAR (Win32 Event) 

ACC_STATE_CHANGE (Accessibility Event) 
WMKEYUP (Win32 Event) 

Further details regarding various preferred mechanisms for tracking user events in 
accordance with the present invention are described below with reference to FIG. . 
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Data Analysis and Display 
FIG. 13 illustratively represents the distributed architecture of a data analysis and 
display subsystem 110 operative to collate and analyze the usage data cooperatively collected 
by client monitoring modules 20. As a product of this analysis, the subsystem 110 generates 
30 various metrics relating to user productivity and also creates associated reports. The 
subsystem 110 includes a data analysis toolkit ("DAT") 114 incorporating a reports engine 
116. The DAT 1 14 is resident on a client computer 12 and communicates with the DBOB 74 
on the server computer 14. The DBOB 74 provides access to the collected monitoring data 
within database 80 which is operated upon by the subsystem 110. The database 80 also 
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5 contains templates for creating reports generated by the subsystem 110, and includes 
embedded SQL commands for supporting queries concerning the stored monitoring data. 

The DAT 1 14 provides the means to view and compare the data collected by the client 
monitoring modules 20. The DAT 114 uses the DBOB 74 to access the monitoring data 
within database 80. In addition, the reports engine 116 of the DAT 114 utilizes an Open 
10 Database Connectivity ("ODBC") connection to the database 80 in connection with data 
retrieval (FIG. 13). The following type of usage information may be generated by the DAT 
114 on the basis of the monitoring data collected and reported by the reported by a specific 
client monitoring module 20: 

• Time Used: The time an application and/or a specific part of an application is 
15 actually used. This calculation is based upon receiving user input and/or 

updated events associated with the application/field. 

• Number of Uses: A count of how many times a specific part of an application 
is used. 

^ • Sequence of Use: Utilization events are time-stamped in order to show the 

jjj 20 sequence of how applications and parts of applications were used. 

• How Used: User input events associated with an application or field in an 
\i application are stored in order to record an exact history of how the application 
pj or field was used. This capability encompasses the state of a field or control 
yd following utilization by a user. The type of state information collected is 
SI 25 dependent upon the nature of the control being monitored (e.g., the state of text 
» field is determined by the associated text, and the state of a "check box" is 

C3 either checked or unchecked). 

Qn 
n 

ru 

o 
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In addition to facilitating acquisition of the type of usage information outlined above, the 
DAT 114 allows the monitoring data collected by the client monitoring modules 20 to be 
queried in a user-defined manner. This feature enables generation of usage reports focused 
upon user-defined usage parameters. In a preferred implementation the DAT 114 is also 
capable of providing various representations of historical usage activity for individual users. 
An exemplary set of such presentations are described below: 

• Usage Histogram: The aggregate time which application invoked by a user within a 
predefined time period was actually used is displayed. Also displayed is the aggregate 
idle time associated with each application (i.e., the period of user inactivity associated 
with each application). 

• Usage Comparison: A comparison of the usage histograms associated with a given 
user over multiple periods of time (one day vs. another, or morning vs. afternoon). 

• Application Usage: A display indicative of the extent of usage of particular features of 
a given application. 

The DAT 114 also preferably allows comparison of the histograms or other usage 
representations of two or more users through, for example, superimposing the histograms or 
representations on the same portion of a display screen. Each of the compared histograms or 
representations may constitute an average taken over many users. 

The DAT 114 provides representations of the activity history associated with specific 
applications. This type of representation will typically be generated by the DAT 114 on the 
basis of the monitoring data collected from a group of users and reported by the client 
monitoring module 20. Set forth below are an exemplary set of such representations specific 
to a particular application: 

• Usage Histogram: For a given duration of time the total usage of the application is 
displayed. Displayed by time increment (1 hour). The number of active users during 
the increment and the percentage of actual usage are displayed. 

• Usage Comparison: Comparison of the usage histograms for an application 
corresponding to different time periods (one day vs. another, or morning vs. 
afternoon). 

• Application Usage: Information is displayed which is indicated of the average use of 
various application features by the monitored group of users. 

Data Management Subsystem 

Referring to FIG. 14, a data management subsystem 120 is disposed to store persistent 

objects used by other components of the inventive data monitoring system. The data 
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management subsystem 120 includes the DBOB 74 and a database 80 having a schema 
divided into several interrelated logical sections. Specifically, the database schema 80 
includes a monitoring data section 124, a profile data section 126, and a report templates 
section 128. The DBOB 74 is operative to access the sections of the schema 124, 126 and 
128 using an ODBC interface 130, thereby allowing the DBOB 74 to remain independent of 
the specific implementation of the database 80. 

A primary purpose of the DBOB 74 is to deliver appropriate objects to other 
requesting system components and to act as a common gateway for the relational database 80. 
In a preferred implementation a single DBOB 74 serves many instances of server module 22 
20 (potentially on separate platforms), and each such DBOB is capable of accessing the 
database 80. This arrangement permits maximum utilization of certain finite resources, such 
as ODBC connectors. The interfaces of the DBOB 74 define the manner in which data within 
the database 80 is manipulated. In providing access to, and managing the database 80, the 
DBOB 74 isolates the low-level storage and methods used to access the database from the 
client monitoring modules 20 and other "clients" of the DBOB 74. 

System Administration 

FIG. 1 5 provides a representation of the architecture of an exemplary implementation 
of a system administration subsystem 140 of the present invention. The system administration 
subsystem 140 is responsible for the central administration of the monitoring system of the 
present invention. It is responsible for configuration and control of the monitoring process. 
The subsystem 140 includes an administration console 144, preferably realized as a COM 
object with a user interface, installed on a particular server computer 14. These "server side" 
components of the subsystem 140 communicate with each other via a hierarchical 
arrangement of server module 22s 20, with server module 22 20a functioning as a superior 
server and the remaining server module 22s 20b acting as subordinate servers. This 
hierarchical configuration can parallel the relationship between work groups in an enterprise 
being monitored in order to facilitate enterprise-wide data collection and monitoring. 

The administration console 144 is disposed to display an active client list, which 

allows the system administrator to monitor usage of the client computers 12. The 

administration console 144 will preferably allow a system administrator to determine if the 
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client service 50 associated with a particular client computer 12 is installed and running. In 
addition, the administration console 144 is capable of causing a new monitoring profile 76 to 
be sent, by way of the applicable server module 22 20, to a client service 50 specified by a 
system administrator. The client monitoring agent 52 associated with this client service 50 
then begins collecting usage data in accordance with the new profile. 

Referring again to FIG. 15, the hierarchical arrangement of server module 22s 20 is 
intended to ensure that each server module 22 20 serves only that number of client monitoring 
modules capable of being adequately supported in light of the processing power of the host 
server machine. By configuring the servers 20b to be subordinate to the server 20a, a tree 
structure can be developed so as to mirror a desired parameter (e.g., network topology, 
business sectors, task groups). In this way management of certain "legs" of the tree can be 
delegated to subordinate administrators, while at the same time allowing oversight by other 
administrators with the requisite permission. When a hierarchical server arrangement is 
employed, the following operations will preferably be carried out recursively throughout the 
arrangement: 

• Start Monitoring 

• Stop Monitoring 

• Reload Profiles 

• Sync clients with their server module 22 

• server module shutdown 

• Flush collected monitoring data to server module 

• Flush client events to server module 
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5 Methods of Restricting Application and Feature Usage 

As was discussed above, the client monitoring agent 52 generally uses one or both of 
two primary methods of collecting information relating to usage of application programs. In 
particular, a first technique involves injecting Hooks DLL into each application program that 
has been activated by the user. The Hooks DLLs are monitor prescribed Windows Hooks in 
10 the application programs and desktop environment so as to determine when specific user input 
events have been performed. In a second approach, information concerning usage of certain 
application programs may also be collected via Accessibility APIs. These Accessibility APIs 
allow more extensive information to be collected regarding usage of various application 
features. 

15 The present invention also contemplates utilizing these methods of tracking 

application feature usage to restrict usage of application features. Advantageously, such 
q restriction may be effected without modifying the application program in which restriction is 

*! to take place. Use of certain application features, or even access to entire application 

yi 

C9 programs, can be restricted on the basis of individual user identity or group membership. As 

'"•'v., 3 

20 is discussed below, the ability to recognize the startup of an application program and the 

yu pending activation of an application feature via injected Windows Hooks and AppFeature 

y i 

- a DLLs provides an opportunity to restrict activation of the feature. 

jyj FIG. 16 is a flowchart representative of the process of restricting application feature 

Q usage in accordance with the invention. In an initial step 170, one or more application 

ru 

p 25 features are declared by the system administrator to be restricted within the application profile 
w applicable to the application program being monitored. This is illustrated by FIG. 17, which 

depicts a view in the profile builder of an application profile containing restricted features. 
When the application program is started, Hooks DLLs and AppFeature DLLs are injected into 
the running process and the applicable application profile is applied (step 174, FIG. 16). If 
30 the application profile indicates that certain features have been restricted, the window 
hierarchy of the running application is examined in order to identify restricted features. This 
identification may be effected in the same way as features specified by the application profile 
to be monitored are identified. Namely, a topmost Accessible Object within the available tree 
of Accessible Objects is obtained and evaluated to determine if it corresponds to the feature 
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being restricted (step 176). The tree of Accessible Objects is then recursively descended until 
an Accessible Object is found which corresponds to the restricted feature (178). When the 
Accessible Object corresponding to the restricted feature is identified, the associated window- 
based control of the running application program is also identified (step 180). The identified 
control may then be disabled (step 182) using the Microsoft API call 
Enable WiNDOW(wmdow/D, False, The window-based control is disabled, and the 
associated feature restricted, in the sense that the control is no longer capable of receiving 
user input (e.g., mouse clicks or keyboard entries). A control disabled in this manner 
typically appears "grayed-out", and cannot be activated by a user. 

Usage of "transient" controls may also be restricted. Examples of transient controls 
are the buttons upon dialog boxes, which are created and destroyed in connection with each 
display of the dialog box. By trapping the user event indicating creation of a dialog, the 
recursive feature detection described above may be applied to disable the newly created 
control (i.e., the dialog box button). 

Dialogs themselves, as well as other top-level windows of application programs, may 
also be restricted. Specifically, creation of the Accessible Object corresponding to the top- 
level window to be disabled is detected using Hooks DLLs as described above. This top-level 
window is then immediately closed and a dialog displayed indicating that use of the window 
is restricted. In order to avoid interfering with the timely processing of events, the dialog 
informing the user of the feature restriction is preferably displayed using a separate, dedicated 
process. 

The feature restriction capability of the present invention may also be invoked to 
ensure that application programs permit only certain desired usage paths. Such desired usage 
paths may be developed by, for example, using monitoring data collected in the manner above 
to identify patterns of efficient usage. Once such patterns have been identified, usage of all 
unrelated features is restricted. Alternatively, a user could be "guided" through operations 
comprised of multiple steps. This is done by permitting the application program to enable 
only those features relevant to a current task to be completed. By limiting the set of features 
available during each task, the navigation of complex application programs offering many 
feature options may be appreciably simplified. 
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Monitoring User Interaction with the World Wide Web 

In another aspect of the present invention, the monitoring technology described herein 
may be embedded in an internet browser on a client machine so as to enable tracking of user 
interaction with the browser. Usage data collected by the client monitoring agent 
incorporated within the browser would be transmitted to the location of the server module 22 
via an HTTP protocol POST operation. The URL to which the data was posted could 
reference a Common Gateway Interface ("CGI") script disposed to read the posted monitoring 
data and forward it to the applicable server module 22 using the data notification scheme 
previously described. 

A different approach may be taken to facilitate the monitoring of web-based 
applications built with JAVA, HTML, DHTML, Java Script, XML and the like delivered to 
host browsers via the world wide web (the "Web"). In this approach the client monitoring 
technology ("Collection Agent") is embedded within the HTML document provided to the 
host browser via the web. By configuring the Collection Agent as a Dynamic Link Library 
("DLL") rather than as an independent executable program, its functionality can be packaged 
as an ActiveX Control or Netscape Plug-in or the equivalent. A reference to the control or 
plug-in is preferably embedded directly within the HTML document provided to the host 
browser. When the HTML document is fetched and rendered by the host browser, the object 
reference to the plug-in or control is resolved and the appropriate installation process begun. 
Depending on the current security settings in use on the client machine, the user may be 
unaware that a Collection Agent in the form of an ActiveX control has been installed. 
However, installation Collection Agents realized as certain types of plug-ins (e.g., a Netscape 
Plug-in) causes generation of a dialog allowing the plug-in to be accepted or rejected.. An 
exemplary sample of HTML source containing a reference to an embedded ActiveX control is 
provided below: 

<P> 

<object ID= M MakeLogin" WIDTH="249" HEIGHT-' 130" CLASSID="CLSID:590669BA- 

3431-1 1D2-BEBB-60505AC 10000" CODEBASE= M http://www.A- 

SoftTech.com/ActiveX/ASoftMakeLogin.CAB#version=l,0,0,5 n > 

</object> 

</p> 
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By embedding reference to a Collection Agent only within Web pages desired to be 
monitored, targeted monitoring of specific Web pages may be effected without undertaking a 
mass installation of monitoring software within each individual client. The targeted 
monitoring of Web pages via a downloaded Collection Agent is of significant potential value 
to entities engaged in "e-commerce" and "e-business". For example, an installed Collection 
Agent can provide nearly instantaneous feedback concerning the manner in which visitors 
interact with the applicable site. Such feedback may indicate: (i) whether users encountered 
any difficulties in utilizing the site, (ii) whether users hesitated while performing a given 
process or task, (iii) whether users abandoned an operation or task, (iii) to what sites users go 
upon leaving the monitored site, and (iv) the manner in which users leave the site (e.g., via 
advertisement selection, other URL, text typed in the GOTO box, etc.). Information of this 
type is generally incapable of being derived from the logs conventionally maintained by web 
servers. These logs typically include information relating only to requests for, and postings 
of, documents. 

Usage data collected by the Collection Agent may be provided to a server module 22 
as described above. That is, the usage data would be transmitted to the location of the server 
module 22 via an HTTP protocol POST operation. A CGI script at the URL to which the data 
is posted would then read the posted monitoring data and forward it to the applicable server 
module 22 as previously described. 

In accordance with another aspect of the present invention and as is more fully 

described herein, user interaction with the world wide web is tracked via monitoring 

mechanisms embedded within web pages downloaded to the user's browser. The web pages 

provided to the user's browser are preferably modified at the server to include scripting 

functions disposed to gather events and otherwise measure user activity while the web page is 

active on the client browser. Measured information may then be provided back to the server 

through one of several mechanisms, including hidden form fields and encoded URLs. 

Alternatively, a Java applet may be utilized to convey measurement information to the 

applicable server. In a preferred implementation the monitoring scripts are embedded within 

web pages using the JavaScript scripting language. Since JavaScript can be embedded within 

the page itself in text form, this implementation advantageously tends to have little or no 
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impact on the time required to download web pages. In addition, JavaScript possesses built-in 
features for accessing all of the components of a web page and interacting with them. This 
concept is more completely described in the section below entitled "Document Object Model 
(DOM)". Finally, unlike an ActiveX control or Netscape plugin, JavaScript isn't able to 
interact with a user's desktop or files system. This characteristic of JavaScript may address 
the concerns of certain users with regard to the activation of monitoring mechanisms on their 
respective client machines. 

Using an application similar to the profile builder 72 described above, a monitoring 
profile is generated designating which features should be tracked and which measures are of 
particular interest. The monitoring profile is then used to "instrument" (i.e., add JavaScript 
instructions to) the web page documents with which the user will interact. This 
instrumentation of web pages is performed independently of the content development process, 
and can even be effected in real time as the page is transmitted to a requestor. The only 
corresponding instrumentation required at the server is incorporation of the applicable 
monitoring script in the page just after the <BODY> tag and in the page header. The general 
format of a web page instrumented in accordance with the present invention is set forth 
below: 

<BODY BGCOLOR=blanchedalmond topMargin-50> 
<SCRIPT LANGUAGE^" JavaScripts 
window.onload = startmonitor; 
</SCRIPT> 

An example given below describes the adding of the onload property to the document in order 
to complete the instrumentation and start the monitoring of a web page. 

In cases where the web pages to be instrumented are dynamically constructed via CGI 
execution or completion of a document template by way of a data base query, the 
instrumentation will often be integrated into the server setup and web page construction 
process at the applicable web site. This integration of instrumentation into each web page 
may be straightforwardly effected by using the "server side include" operations currently 
supported by nearly all commercial web servers. Such operations allow a reference to a 
document to be embedded in other documents. When a document is served to a client the 

32 



p 



5 reference is resolved and additional content written from the other document is incorporated 
within the data stream provided to the client. This avoids the necessity of issuing secondary 
requests from the client to fetch the contents of the reference. 

Transmitting the feature activations collected by an instrumented web page to the 
applicable server via the Internet poses several problems. First, in order to ensure passage 
10 through intervening infrastructure (e.g., firewalls and routers), the connection must be made 
on an open port. One approach to ensuring such passage is to use port 80, the HTTP protocol 
port. However, actually sending the data through this port is complicated by the fact that 
JavaScript does not possess any network privileges. Accordingly, a JavaScript instruction 
cannot simply open a socket connection to the data host on port 80 and transmit the data using 
15 the PUT operation of the HTTP protocol. Although the script could control an applet for 
creating the socket, applets are limited to establishing connections only to the server from 
which they were downloaded. In addition, the time required to download an applet also 
makes this solution unattractive. Instead, in a preferred embodiment two indirect means of 

yn 

jH data transmission using JavaScript are contemplated; namely, transmission via "cookies" and 

~j 20 "hidden FORM fields". 

yJ Cookies are often used to give the illusion of persistence over the connectionless and 

s stateless HTTP protocol. They were originally created to support CGI programming and are 

p implemented as an extension to the HTTP protocol. The data in cookies is automatically 

Q transmitted between the browser and the HTTP server without the knowledge of the user. As 

q 25 is described further below, a cookie consists of a named value and four optional properties for 
w controlling the lifetime, visibility and security of the cookie. The cookies associated with a 

particular page are made available to JavaScript through the cookie property of the document 

object. 

Transmittal of monitoring data can also be performed using what is commonly 
30 referred to as a "hidden FORM". Such a hidden FORM is constructed using elements of a 
FORM defined with <INPUT TYPE=hidden>, which are not visible to the user. A FORM 
containing a single hidden element can be used to transmit data to a server without altering 
the appearance of the web page in which it is embedded. A simple example of a hidden 
FORM is shown below: 
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<FORM NAME-myform ACTION="/cgi-bin/inputdata.pl" METHOD=GET> 

<INPUT TYPE=hidden NAME="mydata"> 

</FORM> 

10 A script may refer to the data item as myform.mydata and set the value to a string containing a 
compressed stream of monitoring data. The METHOD tag in the form definition controls 
whether the data is sent to the server appended to the ACTION URL (GET) or sent in the 
body of the HTTP request (POST). Examples are provided blow expressed in HTTP protocol 
notation. 
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GET http://www.limesoftxorn/cgi-bin/inputdata.pl?mydata ::::: <data > HTTP/1 . 1 



POST http://www.limesoft.com/cgi-bin/inputdata.pl HTTP/ 1.1 
.... [additional header entries here] 
20 mydata=<data> 



The "hidden FORM" approach to data transmission has certain advantages relative to data 
transmission using cookies. For example, in the hidden FORM approach the instrumenting 
m 25 script enables control over precisely when data is sent. Specifically, the data may be sent to 
^ the server at any time by invoking the submitO method of the FORM. This is significant 

a since it is preferable to send monitoring data to the server during times when the user is not 

S initiating any other server requests in order to avoid the impression of degraded performance. 

O The data should preferably be sent during periods of inactivity (e.g., when a user pauses to 

Q 30 view text or images after a document is loaded). 

Real-Time User Help 
Usage data collected by a Collection Agent or client monitoring agent can form the 
basis for providing essentially real-time assistance to end users. This may be effected by 
35 passing usage data through a structure (e.g., a decision tree or similar model) designed to 
detect situations where a user is in need of assistance. User behavior evidencing long pauses 
in certain fields, rapid switching among fields, repeated changing of control values, or aimless 
searching for features are likely indicators of user distress. Under such conditions a short list 
of subjects related to a best guess at the user's intention would be created and provided to the 
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user. The user would have the opportunity to select an intention from the list, and optionally 
could indicate that operations associated with the selected intention be automatically carried 
out. 

Since in certain implementations the collected usage data may only be intermittently 
reported to a server module 22, the above analysis may be conducted at the client In order to 
enable real-time assistance to be provided continuously. The module responsible for 
delivering such functionality ("Help Module") is preferably implemented as an adjunct to the 
standard client monitoring agent. The Help Module informs the client monitoring agent of the 
data items needed to discern a user's action/inaction as it occurs. The Help Module need not 
be highly integrated with the client monitoring agent, and may simply be disposed to receive a 
stream of specified data items. 

Profile Management: Client Side Components 
FIG. 18 illustratively represents the profile builder 72 as included within the profile 
and user management subsystem 70. The profile builder 72 preferably (i) creates and 
manages the Profile Data Objects and their relationships, and (ii) provides an interactive 
means to create Application Profiles. The profile builder 72 builds Application Profiles by 
"pointing and clicking" within the running application to specify features to track. The profile 
builder 72 is preferably packaged as a Win32 based application. The server 14 profile builder 
72 

The profile builder 72 manages five main object types via its user interface (FIG. 19): 
(i) Users, (ii) Groups, (iii) Monitoring Profiles, (iv) Application Profiles, and (v) Product 
Version Groups. Administrative users (simply referred to as users throughout the rest of this 
section) will need to manage individual objects and also their relationships. The class 
diagram of FIG. 20 illustrates these objects and their relationships. From within the Profile 
builder, users can navigate the objects that populate the database 80 using the primary tree 
view (FIG. 21). Selected items are displayed on the right side of the user interface. 
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The profile builder 72 supports different views of the database 80. The view displayed 
is dependent on the type of object selected in the tree view. Selection of a root node in the 
tree view will present a list of all items contained beneath that node. In the "User's view" of 
FIG. 21, the root level group object has been selected by a user, with the appropriate view 

10 displayed on the right. Similar list views for each of the profile object types are obtained via 
user selection of the corresponding root node. Selection of a non-root node profile object will 
display the specific properties for that object. 

From within the User's view, a system administrator is able to do the following: (i) 
modify the name of the user, (ii) modify the user's group, and (iii) explicitly assign the user's 

15 monitoring profile. This will result in overriding of the monitoring profile obtained through 
the user's group. 

FIG. 22 illustratively represents the view afforded through selection of a "Group" tab 
jfl of the user interface for the profile builder 72. The following operations are available upon 

y \ 

gg such selection of the Group tab: (i) modification of a Group's name, (ii) add or remove users 

J"1 20 from a Group, and (iii) assign the monitoring profile for the Group. 

FIG. 23 depicts the view available through selection of a "Monitoring Profile" tab of 
the user interface for the profile builder 72. Such selection of the Monitoring Profile tab 
enables performance of the following operations: (I) modification of a Monitoring Profile's 
Q name, (ii) modification of reporting information for the Monitoring Profile, (iii) addition or 

p 25 removal of Application Profiles from a Monitoring Profile, (iii) explicit assignment of users to 
U a Monitoring Profile, and (iv) assignment of groups to the Monitoring Profile. As is indicated 

by FIG. 23, the Monitoring Profile view utilizes five property pages to perform the functions 
listed above: General, Application Profile List, Group List, User List, and Subnet Mapping 
Profile List. 

30 FIG. 24 illustratively represents the contents of an "Application Profiles" view 

obtained through selection of an Application Profile tab of the user interface for the profile 
builder 72. Selection of this tab enables: (i) modification of the name of an Application 
Profile, (ii) assignment of the specific application upon which the profile is based, and (iii) 
addition/removal of features to an Application Profile. 
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The profile builder 72 makes use of the database object broker 74 to manipulate 
profile database objects. FIG. 25 depicts various interfaces of the database object broker 74 
used to access and manage profile data. Referring to FIG. 25, the profile builder 72 accesses 
the actual monitoring profile objects via the IProfileObjects interface. The accesses typically 
result in enumerated lists of interfaces to specific profile data objects. The database object 
broker 74 holds and maintains these objects, constructing the objects from data contained in 
the profile database system. The profile builder 72 uses the interface to pull information from 
the object. This also supports changing the object's attributes, since the database object borker 
74 maintains independent copies of the objects for each profile builder. 

Referring again to FIG. 25, the IProfileserver moduleEdit interface is used to commit 
changes to the monitoring profile objects to the Profile Database. The commit is performed as 
a database transaction, which serializes the update of the appropriate tables in the database. 
The database management system is responsible for coordinating multiple accesses to the 
Profile Database. 

The database object broker 74 manages the persistence of, and relationships between, 
several types of objects stored in the Profile database: 

• Users 

• Groups 

• Monitoring Profiles 

• Application Profiles 

• Subnet Mappings 

• Product Version Groups 

Clients of the database object broker 74 will generally interact with objects that are "owned" 
by the database object broker 74. Furthermore, these clients will normally be operating on 
remote machines. The interfaces of the database object broker 74 will preferably be crafted to 
allow remote clients to manipulate the objects as efficiently as possible. Nonetheless, the 
database object broker 74 operates to protect the internal integrity of the Profile Database at 
all times. 

All profile data is preferably housed in the Profile Database. The Profile Database is 
typically implemented using a relational database. Part of the task of the database object 
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broker 74 is to store the various profile objects (User, Groups, Monitoring Profiles and 
Application Profiles) to the Profile Database. This generally requires the database object 
broker 74 to be capable of translating between relational table rows and COM or C++ objects. 
The database object broker 74 preferably uses ODBC to access the Profile Database. 

FIG. 26 represents the relationships between various objects included within an 
exemplary implementation of the Profile Database. Each object is directly or indirectly 
related to the others, and each inter-object relationship is bi-directional. Some relationships 
are optional as indicated by a cardinality of 0 at either end. The following relationships have 
non-zero cardinality at both ends greater and are required in the exemplary implementation: 

(1 ) Each User must be a member of a Group; 

(2) Each Group must have a Monitoring Profile; and 

(3) Each Application Profile must be assigned a Product Version Group. 
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The relationships depicted in FIG. 26 are further described in Table I. 



Table V 



Relationship 



User and Group 



Description 



Each User must be a member of one and only one Group. This 
relationship is mandatory. A user cannot exist without being a 
member of a Group. Each Group contains from zero to n users. In 
this direction, the relationship is optional; A Group can exist 
independent of Users. 

These rules dictate that when the user's Group is deleted, the user 
must be deleted (or moved to another Group) to ensure database 
integrity. 



User and Monitoring Profile 



o 
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Group and Monitoring Profile 



Each user can optionally have a Monitoring Profile. This relationship 
is optional. If a user does have a specific Monitoring Profile, it 
overrides the profile the user has by virtue of its Group association. 

Each Group must have exactly one Monitoring Profile. The 
relationship is mandatory. A Group cannot exist without a Monitoring 
Profile. Note however, that not all Monitoring Profiles need have a 
Group. A Monitoring Profile exists independent of a Group. 

Application Profile and Product Each Application Profile must be associated with a Product Version 
Version Group Group. The PVG contains the master set of features available for 

tracking via this Application Profile. 
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10 The Profile Database typically includes two "Default" objects: (i) Default Group, and 

(ii) Default Monitoring Profile. In a preferred implementation of the Profile Database these 
objects always exist and cannot be deleted or re-named. These objects allow database 
integrity rules to be followed without forcing cascaded deletes during certain object deletion 
operations. When a group object is deleted, integrity rules dictate that the users assigned to 

15 the group must be deleted also. However, with the introduction of the Default Group it is 
possible to (optionally) re-assign the users to the Default Group rather than delete them. 

The Default Monitoring Profile works similarly. Database integrity rules dictate that 
when a monitoring profile is deleted, the delete must cascade to groups using the monitoring 
profile, since groups are required to have a monitoring profile. With the introduction of the 

20 Default Monitoring Profile object, it is possible to (optionally) re-assign these groups to the 
Default Monitoring Profile, rather than deleting them. 
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FIG. 27 illustratively represents an exemplary Monitoring Profile Database Schema. 
Only the primary key fields, foreign key fields and other key fields are depicted in FIG. 27. 
The schema of FIG. 27 supports all the relationships involving the Monitoring Profile 
Database described above. 

Table VI describes an exemplary set of default responses to error results from COM 
interface methods used by the profile builder 72. Certain exceptions to these responses are 
described on a per method basis in the tables that follow. 

Table VI 



HRESULT Classes 



Default Error Response 



E_FAIL 

E_NOCOM 

E_FAULT 

Table VII describes the responses of profile builders 72 to error results from the 
implementation of the IProfileObjects interface through the database object broker 74. 

Table VII 



IProfileObjects Method Error Response 



EnumUsers ( ) 



EnumGroups ( ) 



EnumMonProf iles ( ) 



EnumAppProf iles (} 



EnumSubnetMappings ( ) 



E_FAIL - 
E_NOCOM - 
E_FAULT - 
E_FAIL - 
E_NOCOM - 
E_FAULT - 
E_FAIL - 
E_NOCOM - 
E_FAULT - 
E_FAIL - 
E_NOCOM - 
E_FAULT - 
E_FAIL - 
E_NOCOM - 
E FAULT - 
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Table VIII describes the responses of profile builders 72 to error results from the 
implementation of the IProfileserver moduleEdit interface through the database object broker 
74. 

Table VIII 



IProfileserver Error Response 

moduleEdit Method 



NewMonitoringProf ile ( ) 


E_FAIL - 




E_NOCOM - 




E_FAULT - 


DeleteMonitoringProf ile () 


E_FAIL - 




E_NOCOM - 




E_FAULT - 


EditMonitoringProf ile () 


E_FAIL - 




E_NOCOM - 




E_FAULT - 


NewApplicationProf ile () 


E_FAIL - 




EJJOCOM - 




E_FAULT - 


DeleteApplicationProf ile ( ) 


E_FAIL - 




E_NOCOM - 




E„FAULT - 


EditApplicationProf ile ( ) 


E_FAIL - 




E_NOCOM - 




E_FAULT - 


NewGroup ( ) 


E_FAIL - 




E_NOCOM - 




E_FAULT - 


DeleteGroup () 


E_FAIL - 




EJMOCOM - 




E_FAULT - 


EditGroup ( ) 


E_FAIL - 




E NOCOM- 




E FAULT - 


NewUser ( ) 


E FAIL- 




E NOCOM- 




E FAULT - 


DeleteUser () 


E FAIL - 




E NOCOM- 




E FAULT - 


EditUser () 


E FAIL- 




E NOCOM- 




E FAULT - 


NewSubnetMapping ( ) 


E FAIL- 




E NOCOM- 




E FAULT - 


DeleteSubnetMapping ( ) 


E FAIL - 




E NOCOM- 




E FAULT - 


EditSubnetMapping ( ) 


E FAIL- 



41 



IProfileserver Error Response 

moduleEdit Method 

E_NOCOM - 
E _FAULT - 

Profile Management: server module Side Components 

The server side components most directly associated with the Profile Management 

Subsystem are the database object broker 74 and the profile database 72. It is one of the 
primary tasks of the database object broker 74 to provide access to profile data for both 
Profile Management and Monitoring. In addition, the Profile Monitoring Objects created and 
edited by the profile builder 72 are stored in the Profile Database via interfaces of the 
database object broker 74. 

User Monitoring: Client Side Components 

Client Service 

FIG. 28 illustratively represent the Monitoring Software components which execute on 
client machines 12. The client service 50 is a Window95 or NT service that is always running 
on the applicable client machine 12. It monitors the applicable client machine 12 for user 
logon and logoff events. At logon events, it will take action to retrieve a user's monitoring 
profile and start up a client monitoring agent for that user. When a logoff event occurs, it will 
shutdown the client monitoring agent for that user and notifies the server. The client service 
50 is implemented as a service that is perpetually available on the client. It is not itself a COM 
component that is managed by the COM. However, the client service 50 is "COM-enabled", 
meaning that it communicates with the client monitoring agent and the server module 22 
through COM interface pointers. The client service 50 creates a COM object to allow the 
server module 22 to communicate with it via COM interface pointers. 

FIG. 29 illustrates the interaction between the client service 50 and the server module 
22 and client monitoring agent. Since each of these objects are COM-enabled, their 
relationship may be completely characterized through description of their respective 
interfaces. The client service 50 uses ISageserver module on the server module 22 to logon 
and logoff the server module 22 and request profiles for the user that has logged on. The 
logon to the server module 22 corresponds to the user logon to the client so that the 
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monitoring is specific to the user on the particular client desktop. The client service 50 
implements the ISageControlSink as the control interface for outgoing notifications from the 
server module 22. Using the server module 22's ISageCallback interface, the client service 
50 registers its IsageControlSink interface to a "Sink" COM object to receive these 
notifications from the server module 22. The control sink is created when the client service 50 
first logs on to the server module 22. 

The client service 50 uses the IClientMonitoringAgent interface of the client 
monitoring agent to initialize and control the monitoring agent. Commands received via the 
ISageControlSink are passed through to the client monitoring agent. These include Suspend / 
Resume Data Collection and Profile Update commands. Also the client service 50 will 
synchronize the client monitoring agent before logging off to ensure that all data has been sent 
to the server module. When the client monitoring agent is initialized it is handed a pointer to 
the ICollectionStatusSink interface of the client service 50. The client monitoring agent uses 
this interface to provide feedback to the client service 50 regarding it's state as well as to 
signal the client service 50 when data is ready for transmission to the server module 22. 
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The foregoing discussion describes the monitoring system in a fully connected and 
steady state mode of operation. As is described below, other relationships between the 
components of the monitoring system are established during initialization and termination of 
such operation. In a preferred implementation the client Service executes on the Client 
continuously, even in the event that a user is not logged onto the client. The client service 50 
watches for certain events in order to initiate a conversation with the server module 22. Table 
IX describes the operations performed by the client service 50 in response to certain events: 



Table IX 



Event Actions Taken by the client service 50 

User Login 1 Connect to server module 22 telling it who logged in. 

2. Retrieve from the server module 22 the appropriate monitoring 
profile. 

3. Start up a client monitoring agent, passing the client the relevant 
profiling information and the ICollectionStatus sink of the client 
service 50. 

4. Sign up as a listener on the appropriate server module 22's 
outgoing interfaces. 

Notice that prior to the user login, the client service 50 had no 
connection to the server module 22. An assumption of this model is that 
if there is no user logged in on a client, there is no information to be 
collected, thus no communication between client service 50 and server 
module 22. 

For a multiuser terminal server where more than one user can log on, 
the above sequence is modified. If another user has logged on to the 
client such that a connection has already been made to the server, then 
the last step (4) is not necessary. 

User Logoff 1 . Resign as a listener on all server module 22's outgoing interfaces. 

2. Shutdown the client monitoring agent (if one is running). The client 
will pass any buffered information to the server module 22 as part 
of its shutdown sequence. 

3. Disconnect from the server module 22. As part of the 
disconnection the client service 50 will tell the server module 22 of 
the logoff event 

4. Watch for the next login event. 

Again this sequence assumes that only one user has logged on and is 
now logging off. In multiuser client environments, steps 1 and 3 are 
performed only when the last user has logged off. 



In addition to watching for system events (in order to notify the server module 22), the client 
service 50 listens to the server module 22's outgoing commands via the callback registry. The 
notifications possible, along with the client service 50's response, are described in Table X. 
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These notifications can only occur while the client service 50 is listening to the server module 
22, which implies that at least one user has logged on. 



Table X 



Server Module to Notification Meaning And Actions Taken by the Client 

Client Service Service 

Notification 

Reread Profile This informs the client service 50 that the profile currently in use by the 

client monitoring agent has been updated, and must be re-read from the 
server module 22. 

In response to this notification, the client service 50 must: 

1 . Request an updated profile from the server module 22. 

2. Pass the updated profile to the client monitoring agent. 

Suspend Data This informs the client service 50 that the server module 22 has 

Reporting determined that it would like the client monitoring agent associated with 

this client service 50 to stop reporting data for the time being. 

In response to this notification, the client service 50 must: 

1. Tell the client monitoring agent to stop reporting, and furthermore to 

stop all data collection. The client monitoring agent will stop all 

monitoring and collection activities. 

Resume Data This informs the client service 50 that the server module 22 would like 

Reporting to resume data reporting. This notification only makes sense when 

paired with a Suspend Data Reporting notification. 

In response to this notification, the client service 50 must: 
1. Tell the client monitoring agent to resume reporting. 

Synchronize This informs the client service 50 to flush cached monitoring data to the 

server. 

In response to this notification, the client service 50 must: 
1. Tell the client monitoring agent to immediately flush any cached 
data to the server module 22 
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Server Module to Notification Meaning And Actions Taken by the Client 



Client Service 
Notification 



Service 



ServerShutdown 



This informs the client service 50 that the server module 22 is planning 
to do a complete shutdown and all connections to the server should be 
dropped. 

In response to this notification, the client service 50 must: 

• Shutdown the client monitoring agent (no reconnect delay is given) 

• Resign as a listener to the server module 22's outgoing interfaces. 

• Release all interface pointers on the server module. 

As part of the notification, the client service 50 will also 
receive a DWORD argument known as a 
ReconnectionAttemptPolicy that determines when the 
client service 50 should attempt a re-connection. The 
possibilities for reconnection are: 

• At the next logon event, and all successive logon events until one 
succeeds. 

• At some periodic interval after a delay of n seconds. This instructs 
the client service 50 to periodically attempt to re-connect to the 
server module 22 until success. The client service 50 only attempts 
re-connections if a user is currently logged on. When a user logs 
off, the client service 50 suspends re-connection attempts until a 
user logs back on. 

Once a server moduleShutDown notification has been given, along with 
the re-connection policy, the client service 50 will continue to use the re- 
connection policy until the next successful connection. After success, 
the re-connection policy is discarded. 



These notifications can be targeted to all or to specific users managed by the client service 50. 
The server moduleShutdown notification results in disconnecting all client monitoring agents 
as well as the client service 50 itself. When reconnected, all current client monitoring agents 
are re-initialized and connected to the server module. 

Control of Client Services 

Control of client service 50 is performed via the ISageControlSink interface: 
interface ISageControlSink: IUnknown 

{ 

HRESULT EnumActiveClientDigest { 

[out] IEnumActiveClientDigest** ppEnumClient) ; 
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HRESULT RereadProf ile ( 

[in] ULONG elems, [in, size_is (elems) , unique] DWORD* loginlDs); 

HRESULT SuspendDataCollection ( 

[in] ULONG elems, [in, size_is (elems) , unique] DWORD* loginlDs); 

HRESULT ResumeDataCollection ( 

[in] ULONG elems, [in, size_is (elems) , unique] DWORD* loginlDs); 

HRESULT SyncData( 

[in] ULONG elems, [in, size_is (elems) , unique] DWORD* loginlDs); 

HRESULT SyncEvents (void) ; 

HRESULT server modul e Shut down ( [in] DWORD delay); 

HRESULT GetNextDataPacket ( 

[in] DWORD Cookie, [in] LPCOLESTR pktName, 
[out] IDataPacket** pPacket) ; } 

Table XI 



ISageControlSink 
Method 



Description 



EnumActiveClientDigest() 



RereadProfileQ 



SuspendDataCollection() 



ResumeDataCollection() 



SyncData() 



SyncEvents() 



server moduleShutdownQ 



Used to retrieve a structure describing each user currently logged 
onto the client machine for which a client monitoring agent has been 
created. 

Informs the server that those clients indicated in the argument list, or 
all clients if none, should reread their profiles. 

Called to suspend data collection by the server module's clients. 
The client service 50 need not release the IDataReport interface that 
it holds on the server module 22, and is not required to stop 
reporting data ready for transport. 

Note that if a user on any client machine logs off during the time that 
data collection has been suspended there is no need for the 
corresponding ResumeDataCollection() notification, and none will be 
is delivered. The client service 50 will handle all future user logins in 
the normal fashion. 

Called to inform the client service 50 that its clients should resume 
data collection. That is, once again its clients should begin 
collection monitoring data. 

Called to request that the client monitoring agents immediately flush 
their current monitoring data cache to and notify the client service 
50. The client service 50 then informs the server module 22 of the 
ready data. 

Informs the client service 50 to send any events in its log up to the 
assigned server module 22 which in turn incorporates them into the 
NT Event Log. 

Called to direct all client service 50s that the server module 22 is 
going down and each client service 50 and client monitoring agent 
should immediately release all interface pointers on it. The method 
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contains a single parameter to inform the client service 50 of the 
reconnection policy. The reconnection policy describes when the 
client service 50 should attempt a re-connection. It could specify a 
periodic interval, or simply at the next login. 
GetNextDataPacket() Called by the server module 22 to get an IdataPacket interface on 

the indicated data packet. The server streams the packet from the 
client machine to the DBOB and then deletes the packet file. 

5 

Client Monitoring Agent 

The client monitoring agent is a COM object supporting only one interface. A 
primary purpose of the client monitoring agent is to monitor a user's actions on a client 
machine and report the monitoring results to the server module 22. In a preferred 
10 implementation, one client monitoring agent is associated with each user logon. For a 
Windows95 or Windows NT workstation, there will typically be at most one client 
monitoring agent active on the client. In the case of a Window's Terminal server module in 
which many users may be logged on, there will generally be one client monitoring agent per 
sQ active (logged on) user. When the user logs off, the client service 50 removes the client 

m 15 monitoring agent. Each client monitoring agent will typically be unaware of the applicable 
server module 22, and will be aware only of its associated client service 50. 

Referring the FIG. 29, The client service 50 uses ISageserver module on the server 
module 22 to logon and logoff the server module 22 and request profiles for the user that has 
logged on. The logon to the server module 22 corresponds to the user logon to the client so 
0 20 that the monitoring is specific to the user on the particular client desktop. The client service 
jj! 50 implements the ISageControlSink as the control interface for outgoing notifications from 

y 

O the server module 22. Using the ISageCallback interface of server module 22, the client 

service 50 registers its IsageControlSink interface to a "Sink" COM object to receive these 
notifications from the server module 22. The control sink is created when the client service 
25 50 first logs on to the server module 22. 

The client service 50 uses the IClientMonitoring Agent interface of the client 
monitoring agent to initialize and control the monitoring agent. Commands received via the 
ISageControlSink are passed through to the client monitoring agent. These include Suspend / 
Resume Data Collection and Profile Update commands. Also the client service 50 will 
30 synchronize the client monitoring agent before logging off to ensure that all data has been sent 
to the server module. When the client monitoring agent is initialized it is handed a pointer to 
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the ICollectionStatusSink interface of the client service 50. The client monitoring agent uses 
this interface to provide feedback to the client service 50 regarding it's state as well as to 
signal the client service 50 when data is ready for transmission to the server module 22. 

In a preferred implementation the client service 50 run continuously on the applicable 
client computer 12 even when a user is not logged on. The client service 50 watches for 
certain events in order to initiate a conversation with the server module 22. Table XII below 
shows the events and the actions taken by the client service 50 for each: 



Table XII 

Event Actions Taken by the Client Service 

User Login £ Connect to server module 22 telling it who logged in. 

6. Retrieve from the server module 22 the appropriate monitoring 
profile. 

7. Start up a client monitoring agent, passing the client the relevant 
profiling information and the ICollectionStatus sink of the client 
service 50. 

8. Sign up as a listener on the appropriate server module 22 s 
outgoing interfaces. 

Notice that prior to the user login, the client service 50 had no 
connection to the server module 22. An assumption of this model is that 
if there is no user logged in on a client, there is no information to be 
collected, thus no communication between client service 50 and server 
module 22. 

For a multiuser terminal server where more than one user can log on, 
the above sequence is modified. If another user has logged on to the 
client such that a connection has already been made to the server, then 
the last step (4) is not necessary. 

User Logoff 5. Resign as a listener on all server module 22's outgoing interfaces. 

6. Shutdown the client monitoring agent (if one is running). The client 
will pass any buffered information to the server module 22 as part 
of its shutdown sequence. 

7. Disconnect from the server module 22. As part of the 
disconnection the client service 50 will tell the server module 22 of 
the logoff event. 

8. Watch for the next login event. 

Again this sequence assumes that only one user has logged on and is 
now logging off. In multiuser client environments, steps 1 and 3 are 
performed only when the last user has logged off. 



In addition to watching for system events (in order to notify the server module 22), the client 
service 50 is listening to the outgoing commands of server module 22 via the callback 
registry. The notifications possible, along with the response of the client service 50 are shown 
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in Table XIII below. These notifications only occur while the client service 50 is listening to 
the server module 22, which implies that at least one user has logged on. 



Table XIII 



Server Module to Notification Meaning And Actions Taken by the Client 
Client Service Service 
Notification 

Reread Profile This informs the client service 50 that the profile currently in use by the 

client monitoring agent has been updated, and must be re-read from the 
server module 22. 

In response to this notification, the client service 50 must: 

3. Request an updated profile from the server module 22. 

4. Pass the updated profile to the client monitoring agent. 

Suspend Data This informs the client service 50 that the server module 22 has 

Reporting determined that it would like the client monitoring agent associated with 

this client service 50 to stop reporting data for the time being. 

In response to this notification, the client service 50 must: 

2. Tell the client monitoring agent to stop reporting, and furthermore to 

stop all data collection. The client monitoring agent will stop all 

monitoring and collection activities. 

Resume Data This informs the client service 50 that the server module 22 would like 

Reporting to resume data reporting. This notification only makes sense when 

paired with a Suspend Data Reporting notification. 

In response to this notification, the client service 50 must: 
2. Tell the client monitoring agent to resume reporting. 

Synchronize This informs the client service 50 to flush cached monitoring data to the 

server. 

In response to this notification, the client service 50 must: 
2. Tell the client monitoring agent to immediately flush any cached 
data to the server module 22 
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Server Module to Notification Meaning And Actions Taken by the Client 



Client Service 
Notification 



Service 



server 

moduleShutdown 



This informs the client service 50 that the server module 22 is planning 
to do a complete shutdown and all connections to the server should be 
dropped. 

In response to this notification, the client service 50 must: 

• Shutdown the client monitoring agent (no reconnect delay is given) 

• Resign as a listener to the server module 22's outgoing interfaces. 

• Release all interface pointers on the server module. 

As part of the notification, the client service 50 will also 
receive a DWORD argument known as a 
ReconnectionAttemptPolicy that determines when the 
client service 50 should attempt a re-connection. The 
possibilities for reconnection are: 

• At the next logon event, and all successive logon events until one 
succeeds. 

• At some periodic interval after a delay of n seconds. This instructs 
the client service 50 to periodically attempt to re-connect to the 
server module 22 until success. The client service 50 only attempts 
re-connections if a user is currently logged on. When a user logs 
off, the client service 50 suspends re-connection attempts until a 
user logs back on. 

Once a server moduleShutDown notification has been given, along with 
the re-connection policy, the client service 50 will continue to use the re- 
connection policy until the next successful connection. After success, 
the re-connection policy is discarded. 



The notifications described in Table XIII can be targeted to all or to specific users 
managed by the client service 50. The server moduleShutdown notification results in 
disconnecting all client monitoring agents as well as the client service 50 itself. When 
reconnected, all current client monitoring agents are re-initialized and connected to the server 
module. It should be understood that the client service 50 controls the client monitoring agent 
via the IClientMonitoringAgent interface, but that the client monitoring agent does not 
interface with the client service 50. 

Relationship of Client Monitoring Agent to Client Service 

The IClientMonitoringAgent interface is used to control the client monitoring, and is 
described below and in Table XIV: 

interface IClientMonitoringAgent : IUnknown 

{ 
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HRESULT Initialize ( [in] TActiveClientDigest* userlnfo, 

[in] ICollectionStatusSink* pCollectionStatusSink, 
[in] IProf ilelnf o* pProfile, 
[out] DWORD* pid) ; 

HRESULT ProfileUpdate ( [in] IProf ilelnf o* pProfile).; 
HRESULT SuspendDataCollection () ; 
HRESULT ResumeDataCollection () ; 
HRESULT Synchronize ( ) ; } ; 

Table XIV 



IClientMonitoringAgent 

Method 



Description 



Initialize () 

ProfileUpdate () 
SuspendDataCollection { ) 

ResumeDataCollection { ) 
Synchronize ( ) 



Called by the client service 50 to pass the ClientMonitoringAgent the 
Monitoring Profile to use. The client digest is also passed so 
identifying information about the client can be included in the data 
packets. 

Called by the client service 50 to instruct the client monitoring agent 
to use an updated profile. 

Called by the client service 50 to inform the client that is should 
suspend data collection. After receiving this message the client 
monitoring agent should flush its cached buffer of monitoring data, 
then stop all collection and reporting until a ResumDataCollection() 
call. 

Called by the client service 50 to inform the client that it should 
resume data collection. This is called after a call to 
SuspendDataCollection. If the client monitoring agent is not already 
in the suspended state this call is ignored. 

Called by the client service 50 to request that the client monitoring 
agent immediately flush its current monitoring data cache. The client 
monitoring agent should respond by calling 
ICollectionStatusSink::Synchronize() to notify the client service 50 of 
the new data packet. 



Data Packet 

Each data packet is fetched from the client service 50 by the server module 22, but is 
built by the client monitoring agent. When the size of the packet being currently assembled 
by the client monitoring agent exceeds the maximum size specified in the Monitoring Profile, 
or the reporting interval specified in the Monitoring Profile is exceeded, the current packet is 
streamed to a file and the client service 50 signaled with 
ISollectionStatusSink::Synchronize(). The client service 50 will report the presence of ready 
data to the server module 22 via IdataReport::SignalPacketReady(). The data packet object 
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5 (an implementor of IdataPacket) shown in the argument list of the server module's 
ISageControlSink::GetNextDataPacket() is defined as: 

interface IDataPacket: IContentSource 

{ 

10 HRESULT EnumDataltems ( [out] IEnumDataltem** ppEnumDataltems) ; 

HRESULT GetHeader{ [out] IDataHeader** ppDataHeader) ; 

HRESULT AddDataItem( [in] TPersistentDataltem* pltem, 

[out] DWORD* retld) ; 

HRESULT GetPacketFileName ( [in] LPOLESTR* lazyFile) ; 
15 HRESULT DeletePacketFile(void) ; } 

Table XV 



3 in 



IDataPacket 
Method 



Description 



EnumerateDataitems 0 Enumerates the instances of TPersistentDataltem contained in the 

packet by reading the data items from the body of the packet. The 
enumeration retrieves the actual data items.. 

GetHeaderO Returns an IDataHeader interface pointer that can be used to read 

the RFC 822 style name value pairs contained in the header. This is 
^ only a pointer to an interface and not the contents of the header. 

n"] AddDataitem ( ) Adds a single TPersistentDataltem to the body of the packet. This is 

one single piece of collected monitoring data. This method is used by 
the client monitoring agent to insert data items into the packet. 

GetPacketFileName {) Fetches the name of the file (if any) from which the state of the data 

packet was loaded. 

DeletePacketFile () Called by the server module 22 to delete the data packet residing on 

the client after its contents have been streamed to the DBOB and 
fy successfully entered into the DBMS. 

iL-J 

20 An IDataPacket contains between zero and "k" TPersistentDataltems. The structure 

TpersistentDataltem provides a means to operate on an individual Dataltem (a record of some 

desktop event), as an object. 

The role of the IDataHeader is to tag the IDataPacket with an RFC822 style header 

containing name value pairs that serve to identify the data. Information such as sample time, 
25 sample sequence number, software version, userid, client IP address, etc. should be included 

here, along with the content-type label of application/x-sagedata. Ultimately, security related 

information such as a certificate could also be placed here. 
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FIG. 30 depicts the organization of an IDataPacket. In or to support streaming of the 
data items, the IDataPacket extends IcontentSource instead of IUnknown. IContentSource 
has a single method: 

HRESULT GetContentStream( [out] IStream** ppStrm) ; 

This method enables the caller to get a stream to which the serialized state of the packet has 
been written. Each Dataltem contained in the DataPacket represents an occurrence on the 
desktop, specified for collection by the Monitoring Profile loaded by the client monitoring 
agent. The Dataltem is reported as a Meta-Event instead of as the raw information generated 
by the Accessibility Callback or Win32 Hook Filter Procedure which trapped the message. 
Condensing raw events into meta events advantageously allows the volume of data to be 
reduced and facilitates a simplified data base schema. 

Data Header 

interface IDataHeader : IContentSource 

{ 

HRESULT EnumProperties ( [out] IEnumString** ppEnumProps) ; 
HRESULT EnumValues( [out] IEnumString** ppEnumValues) ; 
HRESULT SetValue( [in] LPCOLESTR element, [in] LPCOLESTR value); 
HRESULT GetValue( [in] LPCOLESTR element, [out] LPOLESTR* value); 
HRESULT Remove ([in] LPCOLESTR property); 

}; 

Table XVI 



IDataHeader Method Description 

EnumProperties () Enumerates the properties (value names) held in the header. This 

pulls all of the names to the server module from the client agent. 

EnumValues ( ) Enumerates the values for all of the properties held in the header. 

This enumeration corresponds one-to-one to the enumerated 
properties from the above EnumProperties method. 

Setvalue () This inserts a name (property) value pair into the data packet header. 

Both the name and value are strings. 

Getvalue ( ) Gets the value corresponding to the named property. 

Remove ( ) Removes the named property value pair from the header. 



The server module 22 calls ISageControlSink::GetNextDataPacket() on the client service 50 
to pick up the next ready data packet. From IDataPacket the server module 22 gets a content 
stream and streams the packet from the Client to the DBOB 74. The DBOB 74 enters the 
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5 Dataltems into the DBMS 80 and then returns. On a successful return the server module 22 
will call IdataPacket::DeleteFile(). TPersistentDataltem is defined in the IEnumDataltems 
interface as follows: 

typedef struct tagTPersistentDataltem 

io { 

// Type of object stored 

DWORD * type ; 

// Data and data size 
15 DWORD dataSize; 

[size_is (dataSize) , unique] byte* data; 
} TPersistentDataltem; 

20 Note that the Dataltem is a container for an opaque object. Each occurrence on the desktop 
that is recorded has a definition known only to the client monitoring agent and the DBOB 74. 
The modules which handle the data before it is actually entered in the DBMS (client service 
50 & server module 22) are aware only of the TPersistentDataltem. The actual type of data 
item is indicated by the type member and used by the DBOB 74 to cast the data member to 
I'i 25 the appropriate final type. The dataSize member is used only for marshalling the structure. 

Use of an opaque data type keeps the Interface Definition Language simple and the source 
code extensible without violating COM conventions for interface versioning. 



'IKE? 
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User Monitoring: server module Side Components 
Server Module 

The primary functions of the server module 22 are: 

• Support the bootstrapping of the monitoring process by providing monitoring profiles 
to its clients. 

35 • Provide a point of access to other services such as the DBOB 74. 

• Receive the data collected by instances of client monitoring agent via the client service 
50. 

The server module 22 typically runs on a central NT server as part of the Monitoring 
Subsystem. Although the server module 22 appears to connecting clients as if it were always 
40 running, server module 22 may be implemented as a COM object and therefore only needs to 
be executing if at least one client monitoring agent or client service 50 is currently connected. 
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5 COM will arrange to re-start the server module 22 at upon each attempt to establish such a 
connection. The server module 22 maintains the following state information; 

• Users/workstations currently being monitored 

• Client system failures or the untimely death of a client monitoring agent / client 
service 50 

1 0 • Superior server module 

• Subordinate server modules 

By "maintain" it is meant that applicable information persists regardless of whether or not the 
server module 22 is active or not. In addition to serving the needs of its clients, the server 
module 22 also functions as a node in an ordered network of cooperating server modules. In 
1 5 order to keep network bandwidth use to acceptable levels and manage database transactions, 
cooperation among the server modules on the network with a minimum of negotiation and 
communications is desirable. The need to provide centralized and effective management of 
the network also dictates that the servers have limited awareness of one another, 
tfi These considerations have lead to development of the hierarchical network of FIG. 31. 

|£j 20 In FIG. 1 it is assumed that each server module has between zero and "n" clients, a single 

C5 

\| superior, and between zero and "m" subordinates. Though network topology is fixed during 

n? 

Jfi system operations, changing superior and subordinate connections can be used to easily 

realign node connections. 

□ Referring to FIG. 32, an Admin Console serves as the user interface ("UI") for any 

~ 25 given server module 22. There need not be, and typically is not, a one-to-one relationship 
FU between instances of Admin Consoles and server modules. With appropriate permission any 

Admin Console installation may connect to an arbitrary instance of server module 22. 
Through the use of the Iserver moduleAdmin interface implemented by server module 22, a 
user of the Admin Console configures and maintains the server module 22 installation. The 
30 Admin Console UI also provides access to features of the server module 22 reserved 
exclusively for use by the administrator. Operations such as start/stop monitoring, client list, 
etc. are performed directly, or facilitated by, the server module 22 to which the Admin 
Console is currently connected. 

3 5 Relationship of Server Module to DBOB 

56 
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5 FIG. 33 illustratively represents the primary aspects of the relationship between the 

server module 22 and DBOB 74. One of the chief roles of the server module 22 is to act as a 
single point of contact for its clients. This enable a client to retain only the name or IP 
address of its server module. Access to all other public services (e.g., to the DBOB 74) is 
provided through the server module 22. Clients can interrogate the server module 22 to 
10 determine if it directly implements a desired interface. Where the server module acts as a 
broker for services that it does not directly provide (e.g., the Profile Service) a convenience 
method may be provided as part of the ISageserver module interface. In either case the client 
is free to use the Querylnterface method of IUnknown to ask for services (standard COM 
model). Each server module 22 can have its own DBOB 74, which in turn can be connected 
15 to a remote database server. There is nothing that distinguishes multiple DBOB objects other 
than the channel maintained to the database. If the database is centralized, i.e. the same for all 
DBOBs, then the DBOB objects are indistinguishable. In this way, the DBOB functions as a 
y3 server modules' s gateway to the various database systems. Typically only a few DBOB 

' ^ i 

fii instances will be necessary as the server modules act as control points for the flow of data. 

2i 20 Efficient throughput can be achieved in most cases with a single DBOB 74. 



Access Method to Profile Service 

FIG. 34 illustrates the relationship between the Profile Service and various other 
system components. In a preferred implementation the server module 22 is a client of the 
25 Profile Service. When monitoring begins at a client computer 12, the client service 50 calls 
the Logon ( ) method of the ISageserver module interface implemented by server module 22. 
This is in essence an indirect profile request as the monitoring profile to be applied is one of 
the arguments returned by the Login method. The server module 22 consults the Profile 
Service, which replies with the defined profile most specific to the requestor. 



Access Method to Monitoring Data 



57 



5 FIG. 35 provides an illustrative representation of the data report processing effected by 

the server module 22. In a preferred implementation the server module 22 processes data 
collected by instances of the client monitoring agent which it receives through the client 
service 50 via the IDataReport interface. The client service 50 places the information 
describing the ready data packet on a queue owned by the server module 22. A server thread 
10 services the FIFO queue of packet info and for each packet calls back to the client service 50 
on IsageControlSink (GetNextDataPacket) to get an IDataPacket. The server module 22 
streams the data packet to the DBOB 74 which enumerates the data items in the packet and 
enters them into the appropriate tables in the DBMS. Each data item in a packet represents a 
measure or metric extracted by the client monitoring agent. With respect to the addition of 
15 monitoring data to the DBMS, any DBOB using such DBMS may be considered the 
equivalent of any other DBOB using such DBMS. 

Handling of Monitoring Data 

Like profile information, monitoring data is modeled on the network as an instance of 
20 an object. The object implements IDataPacket and is NOT passed by value. The packet 
contains a set of data items, each of which describes an event that occurred on the desktop. 
Access to the data items as well as the header information carried by a data packet is provided 
Q through the IDataPacket interface. 

Q Because data packets may be streamed (the packet object extends IContentSource) 

25 they can also be easily saved to a file (serialized). Serialized data packets existing outside the 
£3 DBMS should be saved with a unique file type, * . pkt. Files of type . pkt shall be given 

the MIME Type application/x-sagedata and be associated with the Data Analysis Toolkit 
(DAT) in the registry. 

When a client monitoring agent has a read packet of data it serializes the packet to a 
30 file and signals it's client service 50 (ICollectionStatusSink::Synchronize()). The client 
service 50 in turn signals the server module 22 (IDataReport: :SignalPacketReady()). The 
SignalPacketReady() method places the packet information on the packet queue and signals 
the thread which services the FIFO queue. 
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5 Upon being signaled the packet processing thread services the packet queue until it is 

empty at which time the thread returns to a dormant state. For each entry in the queue, the 
packet processing thread will call back to the appropriate instance of the client service 50 and 
get the next ready packets IDataPacket interface via ISageControlSink::GetNextDataPacket(). 
The server module 22 treats the packet as an opaque object and streams it to the DBOB. The 
10 DBOB inflates the packet instance using the serialzed state from the stream. The DBOB then 
enumerates the Data Items and enters them into the appropriate DBMS tables. When control 
is returned the server module 22 the return code indicates whether or not all Data Items were 
successfully entered in the DBMS, if so the server module 22 commands the Data Packet to 
delete itself. If packet processing failed or was only paritally successful the packet will be 
1 5 requeued at a later time by the client service 50. 

Note that the server module pulls data from each client rather than each client 
"pushing" data to the applicable server module. This architecture advantageously provides 
M3 for improved data throughput by distributing the tasks of data processing across all 

w i 

gl cooperating objects. 

•r.. : 

m 20 

yj Threading of Monitoring Data 

fin 

* The server module's need to efficiently handle the data reporting transactions initiated 

y by its many clients, requires that the process of loading the monitoring data into the DBMS be 

m 

□ as streamlined as possible. Getting the data into the DBMS represents the biggest factor 

|» s 25 controlling performance, heavily influencing the number of clients that a given server can 
P handle. 

Streamlining the data reporting process begins with the client service 50, which 
employs an asynchronous data signaling scheme. In accordance with this scheme, a data 
packet is not passed by value from the client service to the server module 22. The monitoring 
30 agent passes information identifying the ready packet to the server module 22 via 
IDataReport. This method simply places the packet informat on a queue managed by another 
thread, signals the thread, and then returns. The server module is then free to accept the next 
client connection. The main "packet manager" thread of the server module services each 
packet streaming the packet from the client service 50 to the DBOB. This asynchronous 
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5 reporting scheme with data "pull" prevents the server from being overloaded by many clients 
attempting to report data concurrently. 

Relationship to Client Service 

FIG. 36 illustrates the reflexive client-server relationship between the server module 
10 22 and the client service 50; that is, each entity is both a client and server of the other. This 
relationship is similar to that of the COM "Connection Point Container" relationship, and is 
illustrated by the state diagram of FIG. 37. In this relationship the client service 50 is the 
client and the server module 22 is the server, but the server module 22 also generates events 
for which the client service 50 is the listener (server). The custom interface ISageCallback 
15 serves in place of the standard IConnectionPointContainer/IConnectionPoint scheme for 
performance reasons. Using a custom interface allows the number of round trips over the 
p» network between the two objects to be greatly reduced. Table XVII outlines some sample 

y3 exchanges between server module 22 and the client service 50. 

W\ 

%\ 20 Table XVII 

W Event Actions Taken by the Client Service 

3 User Login 1 . Connect to server module 22 telling it who logged in. 

* 2. Retrieve from the server module 22 the appropriate monitoring 

O profile. 

Of! 3. Based on the profile, start up a client monitoring agent, passing the 

Q client the relevant profiling information, and an interface pointer to 

the server module 22. 
4. Sign up as a listener on the appropriate server module 22 's 

outgoing interfaces. 
Notice that prior to the user login, the client service 50 had no 
connection to the server module 22. An assumption of this model is that 
if there is no user logged in on a client, there is no information to be 
collected, thus no communication between client service 50 and server 
module 22 

User Logoff 1 . Resign as a listener on all server module 22's outgoing interfaces. 

2. Shutdown the client monitoring agent (if one is running). The client 
will pass any buffered information to the server module 22 as part 
of its shutdown sequence. 

3. Disconnect from the server module 22. As part of the 
disconnection the client service 50 will tell the server module 22 of 
the logoff event. 

4. Watch for the next login event. 
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In addition to watching for system events (in order to provide notification to the server 
module), the client service 50 is listening on the outgoing notification interface of the 
applicable server module 22. An outgoing interface may be realized similarly to a JAVA 
listener interface, except the source and sink (listener) objects can be on remote machines. 
The notifications, along with the action taken by the client service 50 are described in Table 
XVIII below. In a preferred implementation these notifications can only occur while the 
client service 50 is listening on the outgoing interface of the applicable server module 22. In 
addition, the client service 50 is configured to listen only during the time a user is actually 
logged in. 

Table XVIII 



Server Module to Notification Meaning And Actions Taken by the Client 
Client Service Service 

Notification 

Reread Profile This informs the client service 50 that the profile currently in use by the 

client monitoring agent has been updated, and must be re-read from the 
server module 22. 

In response to this notification, the client service 50 must: 

1 . Request an updated profile from the server module 22. 

2. Pass the updated profile to the client monitoring agent. 

Suspend Data This informs the client service 50 that the server module 

Reporting 22 has determined that it would like the client monitoring 

agent associated with this client service 50 to stop 
reporting data for the time being. 
In response to this notification, the client service 50 must: 
1. Tell the client monitoring agent to stop reporting, and furthermore to 
stop all data collection. The client monitoring agent will stop all 
monitoring and collection activities. 

Resume Data This informs the client service 50 that the server module 

Reporting 22 would like it to resume data reporting. This 

notification only makes sense when paired with a Suspend 

Data Reporting notification. 

In response to this notification, the client service 50 must: 
1 . Tell the client monitoring agent to resume reporting. 

Syncronize Data This informs the client service 50 that the server module 

22 would like to have any client side cached monitoring 
data flushed to the server module. 
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Server Module to Notification Meaning And Actions Taken by the Client 
Client Service Service 

Notification 

In response to this notification, the client service 50 must: 
1. Tell the client monitoring agent to immediately flush any cached 
data to the server module 22 

Synchronize Events This informs the client service 50 that the server module 

22 would like to have any client side cached monitoring 
data flushed to the server module. 



In response to this notification, the client service 50 must immediately 
send any events in the client side log to the server. 

server This informs the client service 50 that the server module 

moduleShutdown 22 is planning to do a complete shutdown and all 

connections to the server should be dropped. 

In response to this notification, the client service 50 must: 
Shutdown the client monitoring agent. 

Resign as a listener to the server module 22's outgoing interfaces. 
O Release all interface pointers on the server module, 

ijj As part of the notification, the client service 50 will also 

!H receive a packet of data, known as a 

03 ReconnectionAttemptPolicy, which determines when the 

client service 50 should attempt a re-connection. The 
j possibilities for reconnection are: 

te 1 • At the next logon event, and all successive logon events until one 

5 L succeeds. 

P • At some periodic interval. This instructs the client service 50 to 

W periodically attempt to re-connect to the server module 22 until 

p success. The client service 50 only attempts re-connections if a 

ft) user is currently logged on. When a user logs off, the client service 

q 50 suspends re-connection attempts until a user logs back on. 

j». Once a server moduleShutDown notification has been given, along with 

w the re-connection policy, the client service 50 will continue to use the re- 

j connection policy until the next successful connection. After success, 

the re-connection policy is discarded. 
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FIG. 38 illustratively represent the relationship between the server module 22 and the 
client monitoring agent 52. As is indicated by FIG. 38, the client monitoring agent has no 
awareness of the server module 22. 

FIG. 39 depicts the relationship between superior and subordinate server modules 
10 hierarchically arranged in accordance with one aspect of the invention. A given server 
module may have one and only one superior server module. In a non-networked collection of 
server modules, the superior server need not be assigned (likewise for subordinate servers). 
In the case of a cooperating network of server modules (standard deployment), all server 
modules have a single superior with the exception of the master server. Each server is 
15 preferably made aware of its subordinate servers. It is important to note that a server is 
directly aware of all its subordinate servers at all times, but is only directly aware of those 
clients that are currently connected. Since Clients are configured with their assigned server 
module (and not vice-versa), the task of adding new clients to the system is made simpler. 

Linking the servers together in the manner described above produces a network of cooperative 

j 

J 20 servers that can be represented by the tree diagram of FIG. 3 1 . 

{ Networked server modules collaborate through use of the ISageControlSink interface 

described above (the same callback interface implemented by the client service 50). The UI 
] of the Admin Console provides a means to configure the server modules to repeat commands 

r. 

to all subordinate server modules. From any arbitrary server connection, the Admin Console 
25 can propagate commands up to the root server module, which then propagates notifications 
down the hierarchy to the clients via the ISageControlSink interface. This feature provides a 
means to centrally and concurrently administer sets of server modules. In a large deployment, 
the ability to map the network of servers and delegate management of certain areas to sub- 
administrators eases administration. 
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Interfaces 

FIG. 40 provides an illustrative representation of the public interfaces exposed by the 
server module 22. These interfaces are described in the following subsections. 
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Data Reporting Interface (IDataReport) 

The client service 50 has exclusive use of the IDataReport interface. Through this 
interface a client service 50 notifies the server module 22 of ready data collected by its 
Monitoring. The data items contained within a report packet are then inserted into the 
database via the DBOB 74. 



interface IDataReport : IUnknown 
{ 

HRESULT SignalPacketReady ( [in] DWORD CBCookie, [in] LPOLESTR pktName) ; 

} 

15 Table XIX 

IDataReport Method Description 

SignalPacketReady ( ) Called by the client service 50 to notify the server of data ready 

for archival in the database. 



Ml Servicing the Clients (ISageserver module) 

P3 Interface ISageserver module is used by instances of the client service 50 to contact their 

^ 20 assigned server module 22. 

yy interface ISageserver module : IUnknown 

m { 

s HRESULT Logon ([in] OLECHAR wszUserName [MAX_SAGE_NAME] , 

p 25 [in] OLECHAR wszClntHostName [MAX__SAGE_NAME] , 

W. [in] OLECHAR wsz I PMatch [MAX_S AGE_NAME] , 

p [in] DWORD CallbackID, 

IlJ [in] REFIID riid, 

Q [in] DWORD logonTime, 

Q 30 [out , iid_is (riid) ] void** ppv, // profile 

[out] DWORD* logonID) ; 



HRESULT Logoff ( [in] DWORD CallbackID, [in] DWORD LogonID, 
[in] DWORD logoff Time) ; 

HRESULT GetProfile( [in] DWORD CallbackID, [in] DWORD LogonID, 
[in] REFIID riid, [out , iid_is (riid) ] void** ppv) ; 



64 



Table XX 



on 

a 



C 



Isa? 



ISageserver module Description 
Method 



Logon ( ) Called by the client service 50 when a user logs in at the client host. 

The parameters returned are used to initialize the client monitoring 
agent. 

Logoff () The Logoff method is called by the client service 50 when the user 

logs off the client host. The client shuts down the client monitoring 
agent, resigns as a listener of the server module 22's outgoing 
interface and is removed from the list of active clients maintained by 
the server module 22. 

GetProf ile ( ) A convenience method provided to directly fetch a Monitoring Profile. 

Administration/Configuration ([server moduleAdmin) 

Used primarily by the implementation of the Admin Console and other servers modules, the 
^ 10 Iserver moduleAdmin interface is used to configure and control a server module 22 and 

y 

y3 certain behaviors of its clients. The interface also provides generalized features such as: 

m 

tQ • Examine the list of clients currently reporting data 

.J: • Determine those clients assigned to the server 

W • Examine server status information such as errors, amount of data moved, etc. 



1 5 • Fetch and interface pointer on the superior server 
• Enumerate the subordinate servers, etc. 

interface Iserver moduleAdmin: IUnknown 

{ 

typedef struct tagserver moduleStatus 
20 { 

DWORD upSince; //a time_t 
DWORD numActiveClients; 
BOOL repeatsCommands ; 
DWORD packet sQueued; 
25 DWORD packet sProcessed; 

DWORD dataltemsProcessed; 
DWORD dataTransmitErrors; 
DWORD packetQueueHighWater; 

DWORD packetQueueHighWaterTime; // a time_t 
30 } Tserver moduleStatus; 



HRESULT EnumClients( [out] IEnumString** ppEnumClient); 
HRESULT EnumActiveClients ( [out] IEnumString** ppEnumClient); 
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HRESULT EnumSubordinateserver modules ( [out] IEnumString** ppEnumserver 
module) ; 

HRESULT EnumActiveClientDigest ( 

[out] IEnumActiveClientDigest** ppEnumClient ) ; 
HRESULT RegisterAsSubordinate ( [in] LPOLESTR subordinate); 
HRESULT DeRegisterAsSubordinate { [in] LPOLESTR subordinate); 
HRESULT GetSuperiorserver module ( [out] LPOLESTR* superior); 
HRESULT SetSuperiorserver module ( [in] LPOLESTR superior); 
HRESULT Getserver moduleStatus { [out] Tserver moduleStatus* status); 
HRESULT Repeat Commands ( [in] boolean bRepeat ) ; 
HRESULT SetDBObjectBroker ( [in] LPOLESTR dbObjectBrokerHost); 
HRESULT GetDBObjectBroker ( [out] LPOLESTR* dbObjectBrokerHost); 
HRESULT GetDBOBAdmin ( [out] IDBOBAdmin** ppAdmin) ; 
HRESULT Syncserver moduleState (void) ; 

HRESULT Ge t Event LogEnumerator ( [in] LPOLESTR hostname, 

[in] TEventSource source, [out] IEnumNTEventLog** enumEvents) ; 

}; 

Table XXI 



Iserver moduleAdmin Description 
Method 

EnumClients ( ) 

EnumActiveClients () 

EnumSubordina tserver 
modules ( ) 

EnumActiveClientDigest ( 
) 

RegisterAsSubordinate ( ) 

DeRegisterAsSubordinate 
0 

GetSuperiorserver 
module {) 

SetSuperiorserver 
module () 
Getserver 
moduleStatus ( ) 
RepeatCommands ( ) 

SetDBObjectBroker ( ) 

GetDBOb j ectBroker ( ) 

GetDBOBAdmin ( ) 

Syncserver 
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Enumerates the clients that have historically reported their 
monitoring data to this server. 

Enumerates the clients currently reporting monitoring data to this 
server. 

Enumerates the servers which are subordinate to this server. 

Enumerates the active clients in more detail than EnumClients(). 

Called by another server module 22 to register itself as a 
subordinate of the implementing server module. 

Called by another server module 22 to DE-register itself as a 
subordinate of the implementing server module. 

Returns the an Iserver moduleAdmin interface pointer for the 
superior server of this server. 

Deregisters this server as a subordinate of its current superior (if 
any) and registers it as a subordinate of the specified server. 
Returns information outlining the state and general health of the 
server module 22. 

True of False, specifying whether or not to repeat commands 
relayed by a superior server. 

Configures the server module to use the indicated Data Base Object 
broker when reporting data. 

Returns the hostname where the Data Base Object broker currently 
in use resides. 

Fetches the interface used to administer the DBOB associated with 
this server. 

Forces the server to enumerate the active clients and update its 
internal state. Unresponsive clients are removed from the active 



moduleState ( ) client list. 

GetNTEventLogEnumerator Fetches an enumerator which allows the client to fetch all events 
( ) relative to the SAGE System. This is used to fill the log page of the 

Admin Console. 



Subordinate Server Control (ISageControlSink) 

The server module 22 also implements the ISageControlSink implemented by the 
client service 50. This interface introduces a level of polymorphic behavior shared by all 
objects that populate the distribution tree. Iserver moduleAdmin provides navigational 
capability while ISageControlSink allows actions to be applied to any object (client or server) 
in a homogeneous fashion. Though the interface is the same, the behavior for the server 
module is subtly different from that of the client service 50 implementation. Typically the 
events received via this interface are propagated to the subordinate servers and the appropriate 
client services designated in the callback. Whether or not the commands are propagated to 
subordinate servers is controlled by the I Sage Admin interface from an instance of the Admin 
Console. 

Register Server Control (ISageCallback) 

ISageCallback is implemented by the server module 22 to allow clients to sign up as 
listeners on the outgoing interface of the server module 22. ISageCallback provides 
essentially the same semantics as the standard OLE interface IConnectionPointContainer, but 
can be more efficiently implemented. This same interface is used to register subordinate 
servers to receive the same callbacks. 

interface ISageCallback: IUnknown 
{ 

HRESULT SetupCallbacM [in] REFIID riid, 

[in, iid_is (riid) ] IUnknown* pUnk, 

[in] LPOLESTR pszMachineName , 
[in] boolean bAdvise, 
[out] DWORD* pCookie) ; 

HRESULT Advise ([in] DWORD cookie); 

HRESULT UnAdvise ( [in] DWORD cookie); 

HRESULT ShutdownCallBack( [in] DWORD cookie); 

} 
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Table XXII 



ISageCallback Method 



Description 



SetupCallback ( ) 



Sign up the client REFIID as the listener for callbacks and direct all 
callbacks to pUnk. The machine name of the client is given for the 
callback connection. The advise flag indicates that the client should 
be notified when an event occurs (true) or temporarily suspend 
reports (false). 



The method responds with a cookie that identifies the client in 
subsequent calls to the server. 



Advise ( ) 



Sets the advise flag. 



UnAdvice ( ) 



Unsets the advise flag. 



ShutdownCallback ( ) 



Unregister the client as a listener for server callbacks. The server 
permanently removes the client from its listener list. 



Data Analysis: Client Side Components 



Data Analysis Toolkit 

The Data Analysis Toolkit (DAT) provides the means to view and compare the data 
collected by the client monitoring agents. The DAT is independent of the rest of the system 
components (except for the DBOB) and may be installed separately. The DAT packages a set 
of database queries and resulting reports and graphs into a unified user interface that is easy to 
use. As is indicated by FIG. 13, a reporting engine (e.g., a Crystal Reports Engine) is 
embedded in the DAT. This is to enable easy construction of robust reporting capabilities. 
The DAT is primarily a wrapper for the reports engine functioning to enable the selection, 
customization and use of a set of reports templates. The DAT may use one of two paths to the 
monitoring data database; namely, via the DBOB or using ODBC. 

Graphical Presentations by User 

The monitoring system of the invention enables analysts to view a history of activity 
by a single user. This includes the following presentations: 

• Usage Histogram: For a given duration of time the total usage of each application the 
user used is displayed. Also displayed is the idle time (the time a user was logged in 
but performed no system interactions). 

• Usage Comparison: Compare user's usage histograms over multiple periods of time 
(one day vs. another, or morning vs. afternoon). 
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• Application Usage: For a specific application used by the user, the analyst can display 
the data that represents how the user used the application (which portions of the 
application the user used). 

Graphical Presentations of User Comparisons 

The analyst can view a comparison of multiple users to each other, and to the "average 
user". In an exemplary "histogram comparison", the histograms of two or more users are 
overlapped in order to compare the activities of the users. Information such as the type of 
applications used, and for what duration of time, may be compared. One of the histograms 
can represent an average of many users. 

Graphical Presentations by Application 

The user can view a history of activity by a specific application. This includes the 
following presentations: 

• Usage Histogram: For a given duration of time the total usage of the application is 
displayed. Displayed by time increment (1 hour). The number of active users during 
the increment and the percentage of actual usage are displayed. 

• Usage Comparison: Compare an applications usage histogram over multiple periods of 
time (one day vs. another, or morning vs. afternoon). 

• Application Usage: For a specific application the user can display the data which 
represents how the users used the application (which portions of the application were 
used). 

Computed Fields 

The DAT can be used to review significant information items calculated from the raw data 
contained in the monitoring data database, such as: 

• Time Used: The DAT will add up the time an application and/or a specific 
part of an application is actually used. This calculation is based upon receiving 
user input and/or updated events associated with the application/field. 

• Number of Uses: The DAT will count how many times a specific part of an 
application is used. 

• Sequence of Use: The DAT will time stamp utilization events in order to show 
the sequence of how applications and parts of applications were used. 

• How Used: The DAT will store all user input events associated with an 

application or field in an application in order to record an exact history of how 
the application or field was used This capability extends to include the state 
of a field or control after use by the user. The state information is specific to a 

69 



control. A text field has associated text, a check box is either checked or 
unchecked, etc. 

In addition to the metrics described above, the DAT should provide a means to allow the user 
to construct a query against the DBMS. This feature allows the person responsible for 
evaluating the reported data to generate custom reports and graphics. 

Integration with the Reports Engine 

As explained in previous sections, the DAT is primarily a wrapper for the Reports 
Engine (Engine). The Engine is responsible for querying the database 80 and displaying the 
formatted results. The DAT functions only to provide a means to select and customize reports 
and manage the Engine. In a preferred implementation a Crystal Reports Designer, not the 
Engine, constructs report templates. 

Each report template can be used by the DAT to create a report instance. Various 
template "parameters" can be used to customize each report instance. Perhaps the most 
important customization is the range or volume of data to be included in the report. This is 
done by manipulation of the "where" clause of the query that is part of the report. In addition, 
the Engine allows setting of various attributes of a report. Text items such as titles, chart 
types, header and footer items can be customized. However, the template and the associated 
query determine the basic layout and data schema of a report and cannot be changed within 
the DAT. 
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Handling of Data Analysis Reports 

An significant aspect of the DAT architecture is that all report templates, report 
template parameters, and report property page specifications are held within the database 80 
and read by the DAT. This makes it possible to extend the reports supported by the DAT 
simply by adding report specifications to the database 80. Table XXIII lists the components 
included within a report specification: 

Table XXIII 

Component Description 

Report Template A Crystal reports ".rpt" file (represented as a BLOB) in the 

database. The .rpt files are created using the Crystal Reports 
Report Designer Tool. 

Report Params Information stored in the database that defines the settable aspects 

of the associated report template. Settable aspects include report 
title, report query conditions, chart types, etc. The Report Param 
information also defines a mapping between parameters and 
property pages. 

Report Property Pages The Report Property pages are represented in the database as 

CLSID's. Each CLSID indicates a registered instance of an OLE 
property page. 

FIG. 41 depicts the manner in which these report specifications are held within the 
database 80. An exemplary set of steps for adding a new report to the system of the invention 
are set forth below: 

1. Create a new report template the access the database 80 using the Crystal Reports 
Designer and add the report template to the database as a BLOB. 

2. Determine the Property Pages that will are necessary to set the Report Parameters. In 
many cases, the necessary property pages can be re-used from the pool of property pages 
already available. However, in some cases it will be necessary to create a new property 
page or set of pages for the report. Each property page is an OLE property page utilizing 
predefined interfaces to communicate with the DAT. 
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3. Add the Property Page CLSID's to the database and distribute the COM objects for the 
property pages to the client machines utilizing the DAT. 

4. Set up the Parameter Description for the new report. This creates a mapping between the 
report template's settable aspect and the property page from which each parameter can be 
set. 

Exceptions and Error Handling 

Table XXIV below describes the default responses to error results from COM interface 
methods used by the Data Analysis Toolkit. Any exceptions to these responses are described 
on a per method basis in the Tables that follow. 

Table XXIV 



HRESULT Classes Default Error Response 

E_FAIL 

EJJOCOM 

E_FAULT 

On the server side, the DBOB provides access to the Data Analysis Report objects that 
help to define the reports to be generated from the monitored data. Access to the actual 
monitor data is by direct access to the Monitoring Database via the Reports Engine. 

Data Management: Client Side Components 

FIG. 42 highlights the client side components of a data management subsystem in 
accordance with the present invention. Each of the client side components depicted in FIG. 
42 connect to a DBOB via COM interfaces, which are defined as part of the DBOB 
component. 
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5 Data Management: Server Side Components 

FIG. 43 highlights the server side components of the data management subsystem in 
accordance with the present invention. As is discussed below, the server side components 
included within this subsystem include (i) the DBOB, and (ii) an ODBC Database 
Management System used by the DBOB to store persistent object data. The RDBMS is an 
10 external component to the subsystem supporting a standard ODBC API. Essentially all of the 
Data Management is incorporated into the definition of the DBOB as described below. 

DBOB 

The DBOB functions as a gateway to the Profile, Monitoring Data, and Report 
15 Template Databases. The DBOB represents data in each of these databases as independent 
objects. The DBOB provides a set of interfaces that support access and manipulation of these 
objects. The DBOB is ultimately responsible for the consistency of the objects relative to the 
Q data in these databases. The interfaces of the DBOB completely define how the data within 

yfl the databases may be manipulated. 

ft 20 In providing access to, and managing databases, the DBOB isolates the low-level 

^4 

fU storage and methods used to access these from its clients. Clients of the DBOB see the data 

H' . ! 

m objects in a way that is natural to the client, rather than as database rows and columns. 

L Because the Profile builder requires read/write access to the Profile database, and multiple 

S 

fin instance of the Profile builder may exist, the DBOB supports multiple simultaneous writers 

|Ti 25 using a mechanism such as transactions or record locking. 

B Relationship to ODBC Data Base Management Systems 

FIGS. 44 and 45 represent the process of accessing various database management 
systems from the DBOB via an ODBC interface. ODBC allows the DBOB to operate on as 
30 many different storage models as there are ODBC Desktop Database Drivers. The DBOB 
need not contain any special case code to support the requirement that it communicate with all 
major commercial database products. 

The DBOB isolates the use of ODBC to a single module and exposes the profile 
mangement interfaces as well as the data reporting IDataRepository interface. The DBOB is a 
35 public service available through the server module 22. The primary users are the server 
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5 module 22 itself and the Admin Console. The Profile builder and Data Analysis Tool each 
make direct connection to the DBOB for access to the Profile and Report Template Databases. 

The DBOB is primarily useful in connection with profile management, organization of 
data entry, and management of archived data. Queries are carried out via ODBC, using 
standard SQL queries serviced by the SQL server of the DBMS of choice. The DBOB is not 
10 involved in query operations. This makes it possible to employ commercially available 
graphing and analysis tools in the design of the Data Analysis Toolkit (DAT). 

Relationship to Admin Console 

Referring to FIG. 46, the DBOB is managed via the Admin Console. The main tasks 
1 5 involved in managing the DBOB are shown below: 

• Push Detailed Data to Report DB - For performance reasons the raw monitoring data 
is inserted into fully normalized tables. The input schema is separate from the 

n reporting schema from which queries are made by the Data Analysis tool. Through 

the Admin Console the administrator controls the frequency at which monitoring data 
IH 20 is pushed from the input schema to the reporting schema. 

• Truncate Data From Report DB- Monitoring Data can be automatically truncated 
after a specific period of time (in days) or when a specified quantity of data is reached. 
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Relationship to Server Module 

* 25 Referring to FIG. 47, the server module 22 is a client of the DBOB. A server module 

;=? 22 makes two basic demands on the DBOB: 

m 

O • Request for Monitoring Profiles. When the server module 22 requests a Monitoring 

fU Profile, it will pass the username and IP address of the machine for which the profile is being 

0 requested. The DBOB must return the appropriate Monitoring Profile. The Monitoring 

W 30 Profile requested in this way, is a read-only copy. It will not be modified in any way. 

• The ability to archive the collected data in a timely fashion. 

Relationship to Profile builder 

Referring to FIG. 48, the Profile builder makes use of the DBOB to manipulate profile 
35 database objects. Various interfaces of the DBOB are used as shown below. The profile 
builder requires close control over the manipulation of objects contained in the Profile 
Database. In fact, most of the interfaces of the DBOB are expressly designed to support the 
profile builder. The Profile builder requires support for all the following: 

• Creation, modification and management of all Profile Database objects (Users, 
40 Groups, Monitoring Profiles, Application Profiles, and Product Version Groups) 
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• Management of all relationships between all Profile Database objects. 

To support these requirements, the profile builder requires read-write access to data served by 
the DBOB. Furthermore, the DBOB is preferably prepared to handle multiple simultaneous 
profile builder clients as shown in FIG. 49. 

Relationship to Data Analysis Toolkit 

Referring to FIG. 50, one function of the DBOB is to construct and hold COM objects 
for data represented in the reports analysis database. The DBOB supports the definition of 
report templates, parameter descriptions, and property page definitions. It does not perform 
the querying of data for display (reporting) purposes. Data Analysis Toolkit (DAT) queries 
are preferably carried out via standard SQL as supported by the DBMS of choice. Not 
involving the DBOB in DATA query operations makes it possible to employ commercially 
available graphing and analysis tools in the design of the DAT. 

Interfaces 

FIG. 51 depicts a complete set of interfaces used between the DBOB and other objects 
in a preferred implementation. The DBOB preferably supports the following interfaces: 

• IDataRepository 

• IProfileObjects 

• IProfileserver moduleEdit 

• IProfileserver module 

• IDataManagement 

• IDBOBObjects 

• IDBOBObjectEdit 

The DBOB also uses the Iserver moduleAdmin interface of the server module 22. The 
interfaces are explained in the following sections. 

IDataRepository 

The IDataRepository is used put data items collected from client monitoring agents. It is 
defined as follows: 

interface IDataRepository: IUnknown 

{ 

// Give a data packet (streamed) to the data repository for processing 
HRESULT PutDataPacket ( [in] IStream* pDataStream) ; 

} 
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Table XXV 



IDataRepository Method Description 

PutDataPacket ( ) Called by the server module 22 to pass a stream of monitoring data 

to the DBOB for archival in the database. The data records in the 
packet are loaded into the DBMS. 



IProfileserver module 

The IProfileserver module interface is used to retrieve a user's monitoring profile info. The 
definition is a follows: 

interface IProfileserver module : IUnknown 
{ 

HRESULT GetUserProfileInfo( 

[in] OLE CHAR wszUserName [MAX_SAGE_NAME] , 

tin] OLECHAR ws zClnt Host Name [MAX_SAGE_NAME] , 

[in] OLECHAR wszIPMatch [MAX_SAGE_NAME] , 

[in] REFIID riid, [out , iid_is (riid) ] void** ppv, 

[out] DWORD* UserlD) ; 

} 

Table XXVI 

IProfileserver module Description 

Method =====ssxs=ss=== _^ 

GetuserProf ileinfo 0 Retrieve the complete Monitoring Profile for a user. The name and IP 

match uniquely identify the user. The reference interface ID indicates 
the desired interface to be returned by the method. 

The method, IProfileserver module ::GetUserProfileInfo(), is specifically designed to 
provide the server module 22 a means to retrieve the complete Monitoring Profile for a user. 
The DBOB resolves any profile assignment conflicts using the monitoring profile precedence 
hierarchy. It also merges any application profiles so that the features to be monitored are 
unambiguously defined. The resulting user profile info retrieved is current only for the time 
of retrieval. Subsequent changes to the user profile will require another fetch of the user's 
monitoring profile info. In other words, the retrieved COM object held by the DBOB is not 
synchronized with the Monitoring Profile database. 
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IProfileObjects 

The IProfileObjects interface gives clients the ability to enumerate all the possible objects in 
the Profile Database. It is defined as follows: 

interface IProfileObjects : IUnknown 

{ 

// Enumerators using the standard TProf ileObjectDigest 
HRESULT EnumUsers ( [out] IEnumProf ileObjects** ppEnumUsers); 
HRESULT EnumGroups ( [out] IEnumProf ileObjects** ppEnumGroups) ; 
HRESULT EnumMonProf iles ( [out] IEnumProf ileObjects** ppEnumMPs) ; 
HRESULT EnumAppProf iles ( [out] IEnumProf ileObjects** ppEnumAPs) ; 
HRESULT EnumProductVersionGroups ( 

[out] IEnumProf ileObjects** ppEnumPVGs) ; 
HRESULT EnumApps ( [out] IEnumProf ileObjects** ppEnumApps); 
HRESULT EnumMonitoringSchedules ( 

[out] IEnumProf ileObjects** ppEnumMonScheds) ; 
HRESULT EnumHolidaySchedules ( 

[out] IEnumProf ileObjects** ppEnumHolScheds) ; 

// Enumerators using enhanced digest per object type 
HRESULT EnumAppsEx ( [out] IEnumApps** ppEnumApps); 
HRESULT EnumUsersEx ( [out] IEnumUsers** ppEnumUsers) ; 

Table XXVII 

IProfileObjects Method Description 

EnumUsers ( ) Enumerate the users in the monitoring profile database. 

EnumGroups ( ) Enumerate the groups in the monitoring profile database. 

EnumMonProf iles ( ) Enumerate the monitoring profiles in the monitoring profile database. 

EnumAppProf iles ( ) Enumerate the application profiles in the monitoring profile database. 

EnumProductVersionGroup Enumerate the PVGs in the monitoring profile database. 

s () 

EnumApps ( ) Enumerate the Applications in the database. 

EnumMonitoringSchedules Enumerate the monitoring schedules in the database. These control 
( ) when monitoring occurs on the client. 

EnumHolidaySchedules Enumerate the holiday schedules in the database. These control 

when monitoring occurs on the client. 

EnumAppsEx ( ) Enumerate the Applications in the database returning additional info 

not provided by EnumApps(). 

EnumUsersEx () Enumerate the Applications in the database returning additional info 

not provided by EnumUsers(). 

The enumeration methods, IProfileObjects: :EnumXO^(), each provide an enumerator the 
client can use to enumerate over each of the primary object types stored in the Profile 
Database. In each case, the type returned by the method is the same. It is an enumerator that 
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5 delivers object IDs and Object Names for the object type being enumerated. The full 

declaration of IEnumProfileObjects is shown below, 
interface IEnumProf ileObjects : IUnknown 

{ 

typedef [vl_enum] enum tagTProf ileObjectType 

10 { 

PO_USER = 0, PO_GROUP, PO_MON_PROFILE , PO_APP_PROFILE , PO_APP, 
PO_PRODVERGROUP , PO_MON_SCHED , PO_HOLIDAY_SCHED , 
PO_INVALID 
} TProf ileOb j ectType ; 

15 

typedef struct tagTProf ileObjectDigest 

{ 

TProf ileObjectType dwObj ectType; 

DWORD dwOb j ect ID ; 

20 OLECHAR szObj ectName [MAX_SAGE_NAME] ; 

OLECHAR 

szObjectDesc [MAX_SAGE_DESCRIPTION] ; 
} TProf ileObjectDigest ; 

i 25 [local] 



•sea" 

Ul 
00 



HRESULT Next{ [in] ULONG celt, 

[out] TProf ileObjectDigest *rgelt, 
[out] ULONG *pceltFetched) ; 



HI 

yi 30 [call_as (Next) ] 

^ HRESULT RemoteNext( [in] ULONG celt, 

- ; [out, size_is (celt) , length_is (*pceltFetched) ] 

^ TProf ileObjectDigest *rgelt, 

U [out] ULONG *pceltFetched) ; 

E 35 

U HRESULT Skip ([in] ULONG celt); 

: 

O HRESULT Reset () ; 

s 

40 HRESULT Clone ([out] IEnumProfileObjects **ppenum) ; 

} 

As indicated by IEnumProfileObjects::Next(), this enumerator allows the client to enumerate 
a series of structures, TProfileObjectDigest, that contain an Obj ectType, ObjectID, and 
45 ObjectName and ObjectDescription. The ObjectType is an enumeration designating the type 
of profile object. The ObjectID is a unique number that identifies the object. It cannot change 
for the lifetime of the object. The ObjectName is a unique display name shown to human 
users. Although unique, the ObjectName can be changed during the lifetime of the object. 
ObjectDescription is a brief description of the object displayed to human users. 
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Profile Database Object Interfaces 

Objects within the Profile Database are exposed to clients of the DBOB as COM objects. The 
profile database objects, and their interfaces are depicted in FIG. 52. Each of the profile 
database objects is exposed as a COM object to DBOB Clients. Each supports a single, 
10 primary interface containing all possible operations. 

IProfileDatabaseObject 

All Profile Database objects have a name and ID. The name is intended to be displayed to 
users. Although the names are required to be unique within each object type, object names 
can change during the lifetime of the object. An Object's ID is a system-assigned, immutable, 
15 unique (within each type), identifier for the object. The interface below is the base interface 
for all profile database objects. It allows retrieval of the object ID and object name. It also 
allows the name to be changed. 

interface IProf ileDatabaseObject : IUnknown 

{ 



j\ 20 HRESULT GetDigest ( [out] TProf ileObjectDigest* pdigest) ; 

HRESULT SetName ( [in] OLECHAR wszName [MAX_SAGE_NAME] ) ; 
*«j HRESULT SetDescription( [in] OLECHAR wszDesc [MAX_SAGE_DESCRIPTION] ) ; 

P 1 HRESULT Persist ( [out] DWORD* objectID) ; 

} 



n 
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Table XXVIII 



IProfileDatabaseObject Description 
Method 



GetDigest ( ) Retrieve the name and id of the object. 



SetName ( ) Set the name of the object. This will fail if the new name is not unique 

among the object's type. 
SetDescription ( ) Set the description of the object. 

Persist ( ) Saves the current state of the object to the DBMS. 
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IUser 

interface IUser 



IProf ileDatabaseOb j ec t 



{ 



HRESULT GetGroup ( [out] TProf ileObj ectDigest* ppGroup) ; 

HRESULT SetGroup ( [in] DWORD dwGroupID) ; 

HRESULT GetMonitoringProf ile ( [out] TProf ileObj ectDigest* pMProf ile) ; 
HRESULT SetMonitoringProf ile ( [in] DWORD dwMPID); 
HRESULT GetFullName ( [out] OLECHAR desc [MAX_SAGE_NAME] ) ; 
HRESULT SetFullName ( [in] OLECHAR desc [MAX SAGE NAME]); 



Table XXIX 



IUser Method 



Description 



GetGroup ( ) 
SetGroup ( ) 

GetMonitoringProf ile ( ) 
SetMonitoringProf ile ( ) 
GetFullName () 
SetFullName () 



A user object is required, at all times, to have a 



Get the user's group, 
group. 

Modify the user's group. This method will remove the user's current 
group association and add the user to a different group. 
Retrieve the monitoring profile that has been specifically assigned to 
this user, if one exists. It is possible that no monitoring profile has 
been assigned. 

Assign a specific monitoring profile for this user. When set, this 
overrides the monitoring profile the user has by virtue of the user's 
group association. 

Used to retrieve the full user name (Ken W. Smith). In contrast the 
GetName() method inherited via IprofileDatabaseObject would return 
the username (kenw). 
Sets the full username. See above. 



Igroup interface IGroup : IProfileDatabaseObject 

HRESULT EnumUsers ( [out] IEnumProf ileObj ects** ppEnumUsers) ; 

HRESULT GetMonitoringProf ile ( [out] TProf ileObj ectDigest* pMProfile) ; 

HRESULT SetMonitoringProf ile ( [in] DWORD dwMPID) ; 

} 

Table XXX 

IGroup Method Description 



EnumUsers ( ) Enumerate the users that are a member of this group. 

GetMonitoringProf ile () Get the monitoring profile that this group is associated with. Groups 

are required to have an associated monitoring profile at all times. 
SetMonitoringProf ile () Change the monitoring profile used by this group. 
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interface IMonitoringProf ile : IProf ileDatabaseObj ect 

{ 

HRESULT EnumAssignedUsers ( [out] IEnumProf ileObjects** ppEnumUsers) ; 
HRESULT EnumUsingGroups ( [out] IEnumProf ileObjects** ppEnumGroups) ; 
HRESULT EnumAppProf iles ( [out] IEnumProf ileObjects** ppAppProf iles) ; 
HRESULT AddAppProf ile ( [in] DWORD APID); 
HRESULT RemoveAppProf ile ( [in] DWORD APID); 

HRESULT GetSuspendCollectionFlag ( [out] BOOL* pbSuspended) ; 
HRESULT SetSuspendCollectionFlag ( [in] BOOL bSuspended) ; 

HRESULT GetMonitoringBuf ferLimit ( [out] DWORD* pdwNumBytes) ; 
HRESULT SetMonitoringBuf ferLimit ( [in] DWORD dwNumBytes) ; 

HRESULT GetMinReportinglnterval ( [out] DWORD* pdwlnterval) ; 
HRESULT SetMinReportinglnterval ( [in] DWORD dwlnterval); 

HRESULT Get Input SummaryTime ( [out] DWORD* dwlnterval); 
HRESULT SetlnputSummaryTime ( [in] DWORD dwlnterval); 

HRESULT GetMonitoringSchedulelD ( [out] DWORD* dwSchedID); 
HRESULT SetMonitoringScheduleID( [in] DWORD dwSchedID) ; 

} 

Table XXXI 



IMonitoringProfile 
Method 



Description 



EnumAssignedUsers () 

EnumUsingGroups ( ) 
EnumAppProf iles ( ) 
AddAppProf ile ( ) 
RemoveAppProf ile {) 
GetSuspendedCollectionFlag ( ) 

SetSuspendedCollectionFlag ( ) 
GetMonitoringBuf ferLimit {) 

SetMonitoringBuf ferLimit { ) 
GetMinReportinglnterval ( ) 
SetMinReportinglnterval () 
GetlnputSummaryTime () 

SetlnputSummaryTime ( ) 
GetMonitoringSchedulelD () 
SetMonitoringSchedulelD () 



Enumerate the users assigned to this monitoring profile. 

Enumerate the groups using this monitoring profile. 

Enumerate the application profiles that this monitoring profile is using. 

Add an application profile to the current monitoring profile. 

Remove an application from the current monitoring profile. 

Gets the flag which determines whether or not data collection is 

suspended for this Monitoring Profile. 

Sets the collection suspended flag. See above. 

Get the monitoring buffer limit. The monitoring buffer limit is used by 

the client monitoring agent in limiting its event data cache size. 

Set the monitoring buffer limit 

Get the minimum reporting interval. 

Set the minimum reporting interval. 

Get the interval at which input events (mouse & kbd) are 
summarized. 

Set the input summary interval. 

Get the iD of the monitoring schedule in use by the profile. 
Set the ID of the monitoring schedule in use by the profile 
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lAppProfile 

The lAppProfile interface provides access to application and feature information for a specific 
application profile. 
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interface IAppProfile : IProf ileDatabaseObject 

{ 

// The product version group this profile is based upon 
HRESULT GetProductVersionGroupID ( [out] DWORD* pPVGID) ; 
HRESULT SetProductVersionGroupID ( [in] DWORD PVGID) ; 

// Enum all features associated with this profile 

HRESULT EnumFeatures ( [out] IEnumFeatures** ppEnumFeatures) ; 

// Add or Remove a specific feature from the profile 

HRESULT AddFeature ( [in] DWORD featurelD, [in] BOOL bRestricted); 

HRESULT RemoveFeature ( [in] DWORD featurelD); 



// Set the restricted flag (veto) a feature 
HRESULT SetFeatureRestrictedFlag ( [in] DWORD featurelD, 
[in] BOOL bRestricted) ; 



Table XXXII 



IAppProfile Method 



Description 



GetProductversionGroupiD o Retrieve the ID of the PVG with which the App Profile is associated. 



SetProductVersionGroupID ( ) 
EnumFeatures () 

AddFeature () 

RemoveFeature ( ) 
SetFeatureRestrictedFlag ( ) 



Set the ID of the PVG with which the App Profile is associated. 

Enumerate the features tracked by this App Profile. This will be a 

subset of the features available through the associated PVG. 

Add a Feature to this App Profile from the set of features maintained 

by the associated PVG. 

Remove a Feature from the App Profile. 

Flag the feature as being Restricted. This means that it will be made 
unavailable to the user. 



The enumeration method, EnumFeatures(), provides an enumerator to access all features 
associated with the application profile. The full declaration of IEnumFeatures is shown 
below. 

interface IEnumFeatures : IUnknown 

{ 

// Enum to describe the type of feature 
typedef [vl_enum] enum tagTFeatureType 

{ 

FEATURE_ACCESSIBLE = 0, 
FEATURE_ACCELERATOR 
} FeatureType ; 

typedef struct tagFeatureDigest 
{ 

DWORD featurelD; 
FeatureType type; 

OLECHAR name [MAX_SAGE_NAME] ; 

OLECHAR description [MAX_SAGE_DESCRIPTION 
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DWORD role ID; 

OLE CHAR role [MAX_ROLE_TEXT] ; 

boolean restrict; 
boolean canRestrict; 
} TFeatureDigest; 



[local] 

HRESULT Next( [in] ULONG celt, 

[out] TFeatureDigest *rgelt, 

[out] ULONG *pceltFetched) ; 

[call_as (Next) ] 

HRESULT RemoteNext( [in] ULONG celt, 

[out, size_is (celt) , length_is (*pceltFetched) ] 
TFeatureDigest *rgelt, 
[out] ULONG *pceltFetched) ; 

HRESULT Skip ([in] ULONG celt); 

HRESULT Reset ( ) ; 

HRESULT Clone ([out] IEnumFeatures **ppenum) ; 



As indicated by IEnumFeatures ::Next(), the enumerator fills a structure, TFeatureDigest, that 
contains general information relating to the specific feature. Detailed information for the 
feature is obtained through the IFeature interface, as described in the next section. 



IFeature 

The IFeature interface provides access to information obtained through Active Accessibility, 
interface IFeature : IUnknown 

{ 

// Not a Prof ileDatabaseObject . . 

/ / Features are persisted by the parent ProdVerGroup 
// Methods common to all feature types 
HRESULT GetDigest ( [out] TFeatureDigest* pdigest) ; 
HRESULT SetName( [in] OLECHAR wszName [MAX_SAGE_NAME] ) ; 

HRESULT SetDescription( [in] OLECHAR descr [MAX_SAGE_DESCRIPTION] ) ; 
HRESULT GetDescription{ [out] OLECHAR descr [MAX_SAGE_DESCRIPTION] ) ; 

// Feature restriction 

HRESULT SetRestrictionCapability ( [in] BOOL bCanRestrict) ; 
HRESULT GetRestrictionCapability ( [out] BOOL* pbCanRestrict) ; 

// Type of feature: FEATURE_ACCES S IBLE or FEATURE_ACCELERATOR 
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HRESULT SetType([in] FeatureType type) ; 
HRESULT GetType { [out] FeatureType* pType) ; 

// Accessible object methods 
HRESULT SetAccDescription ( 

[in] OLECHAR accDescription [MAX_SAGE_DESCRIPTION] ) ; 
HRESULT GetAccDescription ( 

[out] OLECHAR accDescription [MAX_SAGE_DESCRIPTION] ) ; 
HRESULT SetAccName ( [in] OLECHAR Name [MAX_SAGE_NAME] ) ; 

HRESULT GetAccName ( [out] OLECHAR Name [ MAX_S AGE_NAME ] ) ; - ■ " 

HRESULT SetAccSearchName ( [in] TAccSearchCode searchCode, 

[in] OLECHAR srchName [MAX_SAGE_NAME] ) ; 
HRESULT GetAccSearchName ( [out] TAccSearchCode* pSearchCode, 

[out] OLECHAR srchName [MAX_SAGE_NAME] ) ; 
HRESULT SetUseAccName ( [in] BOOL bUse) ; 
HRESULT GetUseAccName ( [out] BOOL* bUse) ; 
HRESULT SetAccRole ( [in] DWORD rolelD, 

[in] OLECHAR accRole [MAX_ROLE_TEXT] ) ; 
HRESULT GetAccRole ( [out] DWORD* rolelD, 

[out] OLECHAR accRole [MAX_ROLE_TEXT] ) ; 
HRESULT SetUseAccRole ( [in] BOOL bUse) ; 
HRESULT GetUseAccRole ( [out] BOOL* bUse) ; 

HRESULT SetAccWindowClass ( [in] OLECHAR accWindowClass [MAX_SAGE_NAME] ) 
HRESULT Get AccWindowClass ( 

[out] OLECHAR accWindowClass [MAX_SAGE_NAME] ) ; 
HRESULT SetUseAccWindowClass ( [in] BOOL bUse) ; 
HRESULT GetUseAccWindowClass { [out] BOOL* bUse) ; 
HRESULT SetPosition( [in] long xpos, [in] long ypos) ; 
HRESULT GetPosition( [out] long* pxpos, [out] long* pypos) ; 
HRESULT SetUsePosition( [in] BOOL bUse) ; 
HRESULT GetUsePosition( [out] BOOL* bUse) ; 
HRESULT SetSize([in] long width, [in] long height); 
HRESULT GetSize( [out] long* pwidth, [out] long* pheight) ; 
HRESULT SetUseSize ( [in] BOOL bUse) ; 
HRESULT GetUseSize ( [out] BOOL* bUse) ; 
HRESULT SetControlID ( [in] long control ID) ; 
HRESULT GetControlID ( [out] long* pcontrolID) ; 
HRESULT SetUseControlID( [in] BOOL bUse) ; 
HRESULT GetUseControlID( [out] BOOL* bUse) ; 
HRESULT SetAccessibleID( [in] long accID) ; 
HRESULT GetAccessibleID( [out] long* paccID) ; 
HRESULT SetUseAccessibleID( [in] BOOL bUse) ; 
HRESULT GetUseAccessibleID( [out] BOOL* bUse) ; 

// Which accessible attrs are valid for this feature 
// uses the type "TAccUsageFlag" 

HRESULT SetValidAccAttrFlag( [in] DWORD validFlag) ; 
HRESULT GetValidAccAttrFlag( [out] DWORD* pvalidFlag) ; 

// accelerator methods 

HRESULT SetVirtualKeyCode { [in] byte vkCode) ; 
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HRESULT GetVirtualKeyCode ( [out] byte* pVkCode) ; 
HRESULT SetUseShiftKey ( [in] BOOL bUse) ; 
HRESULT GetUseShiftKey ( [out] BOOL* pbUse) ; 
HRESULT SetUseControlKey ( [in] BOOL bUse) / 
HRESULT GetUseControlKey ( [out] BOOL* pbUse) ; 
HRESULT SetUseAltKey ( [in] BOOL bUse) ; 
HRESULT GetUseAltKey ( [out] BOOL* pbUse) ; 



// Feature Ancestor methods 

HRESULT EnumFeatureAncestors ( [out] IEnumFeatureAncestor** ppEnum) ; 
HRESULT AddAncestor ( [in] TFeatureAncestor* ancestor); 
HRESULT UpdateAncestor ( [in] TFeatureAncestor* ancestor); 

Table XXXIII 



IFeature Method Description 

GetDigestO Retrieve the associated TFeatureDigest structure. 

setNameO Set the name for the feature. 

SetDescriptionO Set the human readable description for the feature. 

GetDescriptionO Set the human readable description for, the feature. 

SetRestrictionCapabiiity Specify whether or not the feature can be 

restricted. 

GetRestrictionCapabiiity Determine whether or not the feature can be 

restricted. 

SetType () 
GetType ( ) 

SetAccDescription ( ) 
GetAccDescription ( ) 
SetAccName {) 
GetAccName 0 
SetAccSearchName { ) 
GetAccSearchName { ) 
SetUseAccName () 
GetUseAccName () 
SetAccRole () 
GetAccRole ( ) 
SetUseAccRole () 
GetUseAccRole ( ) 

SetAccWindowClass () 
GetAccWindowClass () 
SetUseAccWindowClass () 
GetUseAccWindowClass () 
SetPositionO 
GetPosition ( ) 
SetUsePosition ( ) 
GetUsePosition { ) 
SetSizeO 
GetSizeO 
SetUseSize ( ) 
GetUseSize () 
SetControlIDO 
GetControlIDO 
SetUseControlIDO 
GetUseControlIDO 
SetAccessiblelD ( ) 
GetAccessiblelD ( ) 
SetUseAccessiblelD ( ) 
GetUseAccessiblelD ( ) 
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IFeature Method Description 

SetValidAccAttrFlag ( ) 
GetValidAccAttrFlag ( ) 
SetVirtualKeyCode ( ) 
GetVirtualKeyCode ( ) 
SetUseShiftKeyO 
GetUseShif tKeyO 
SetUseControlKey ( ) 
GetUseControlKey ( ) 
SetUseAltKeyO 
GetUseAltKeyO 
EnumFeatureAncestors ( ) 
AddAncestor ( ) 
UpdateAncestor ( ) 



The enumeration method, EnumFeatureAncestors(), provides an enumerator to access all 
ancestors associated with the feature. Ancestry information defines a hierarchy of user 
interface windows related to a specific feature. The full declaration of IEnumFeatureAncestor 
is shown below. 

interface IEnumFeatureAncestor : IUnknown 

{ 

typedef [vl_enum] enum tagTAccUseFlags 
{ 

ACC_USE_NONE = 0x0, 

ACC_USE_NAME = 0xl # 

ACC_USE_WINCLASS = 0x2, 
ACC_USE_ROLE = 0x4, 

ACC_USE_POSITION = 0x8, // Use parent -relative position 
ACC_USE_SIZE = 0x10, // Use accessible object size 

ACC_USE_CONTROLID = 0x2 0, // Use HWND control ID value 
ACCJJSE_ACCESSIBLEID = 0x40 // Use accessible ID value 
} TAccUsageFlag; 

// Enum to describe the search method used in feature names 
typedef [vl_enum] enum tagTAccSearchCode 

{ 

ACC_SEARCH_EXACT = 0, 
ACC_SEARCH_SUBSTR , 
ACC_SEARCH_STARTSWITH , 
ACC_SEARCH_INVALID 
} TAccSearchCode; 

typedef struct tagFeatureAncestor 

{ 

DWORD dwUsageFlags ; // Which identity attrs to use 

DWORD dwValidAccFlags; // Which identity attrs are valid 
DWORD dwLevel; 

OLE CHAR wszAccRole [MAX_ROLE_TEXT] ; 
DWORD dwAccRolelD; 

OLECHAR wszAccName [MAX_SAGE_NAME] ; 

OLECHAR wszAccWinClass [MAX_SAGE_NAME] ; 

OLECHAR wszAccSearchName [MAX_SAGE_NAME] ; 

TAccSearchCode nameSearchCode ; 
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long xpos ; 
long ypos; 
long width; 
long height; 
long control ID; 
long accessiblelD; 
} TFeatureAncestor; 



[local] 

HRESULT Next( [in] ULONG celt, 

[out] TFeatureAncestor *rgelt, 

[out] ULONG *pceltFetched) ; 



[call_as (Next) ] 

HRESULT RemoteNext{ [in] ULONG celt, 

[out, size_is (celt) , length_is (*pceltFetched) ] 
TFeatureAncestor *rgelt, 
[out] ULONG *pceltFetched) ; 

HRESULT Skip ([in] ULONG celt); 

HRESULT Reset () ; 

HRESULT Clone ( [out] IEnumFeatureAncestor **ppenum) ; 

}; 

As indicated by IEnumFeatureAncestor: :Next(), the enumerator fills a structure, 
TFeatureAncestor, that contains information relating to the specific feature ancestor. 



IProfileserver moduleEdit 

IProfileserver moduleEdit allows creation, deletion and editing of all profile database objects. 
The complete interface is shown below: 

interface IProfileserver moduleEdit : IUnknown 

{ 

// monitoring profile 
HRESULT NewMonitoringProf ile ( 

[in] OLECHAR wszMPName [MAX_SAGE_NAME] , 

[in] REFIID 

riid, 

[out, iid_is (riid) ] void** ppv) ; 

HRESULT EditMonitoringProf ile ( 

[in] DWORD MP ID, 

[in] REFIID 

riid, 

[out, iid_is (riid) ] void** ppv) ; 

HRESULT DeleteMonitoringProf ile ( 
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[in] DWORD MPID, 

[in] BOOL bReassignGroups) ; 



void** 



// app profile 

HRESULT NewApplicationProf ile ( 

[in] OLE CHAR wszAPPName [MAX_SAGE_NAME] , 
[in] REFIID 

riid, 

[out, iid_is (riid) ] 
HRESULT EditApplicationProf ile ( 

[in] DWORD 
[in] REFIID 

riid, 

[out , iid_is (riid) ] 
HRESULT DeleteApplicationProf ile ( [in] DWORD 



ppv) ; 
APID, 



void** ppv) / 

APID) ; 



// group 

HRESULT NewGroup ( [in] OLECHAR wszGroupName [MAX_SAGE_NAME] 

[in] DWORD 

MPID, 
riid, 

HRESULT EditGroup( 



riid, 

HRESULT DeleteGroup( 
bReassignUsers) ; 



[in] REFIID 

[out, iid_is (riid) ] 

[in] DWORD 
[in] REFIID 

[out, iid_is (riid) ] 

[in] DWORD 
[in] BOOL 



void** 



void** 



ppv) ; 
GroupID, 

ppv) ; 
GroupID, 



// User 

HRESULT NewUser( [in] OLECHAR wszUserName [MAX_SAGE_NAME] 

[in] DWORD 
[in] REFIID 



riid, 

HRESULT EditUser( 
riid, 

[out, iid_is (riid) ] 
HRESULT Dele teUser ( [in] DWORD GroupID); 



[out, iid_is (riid) ] 

[in] DWORD 
[in] REFIID 



void** 



void** 



GroupID, 

ppv) ; 
UserlD, 

ppv) ; 



// Applications 
HRESULT NewApp( 



HRESULT EditApp( 



[in] OLECHAR fingerprint [MAX_SAGE_APP_FINGERPRINT] , 

[in] REFIID riid, 

[out, iid_is (riid) ] void** PP V ) ; 
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[ in] DWORD appID , 

[in] REFIID riid, 
[out, iid_is (riid) ] void** ppv); 

HRESULT DeleteApp( [in] DWORD appID) ; 
HRESULT GetAppIDByFingerprint ( 

[in] OLECHAR fingerprint [MAX_SAGE_APP_FINGERPRINT] , 
[out] DWORD* appID) ; 
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// Product Version Groups 

HRESULT NewProductVersionGroup ( 

[in] OLECHAR name [MAX_SAGE_NAME] , 

[in] REFIID riid, 

[out, iid_is (riid) ] void** PP V ) ; 

HRESULT EditProductVersionGroup { 

[in] DWORD PVGID, 
[in] REFIID riid, 
[out, iid_is (riid) ] void** ppv) ; 

HRESULT DeleteProductVersionGroup ( [in] DWORD PVGID) ; 



O 

yi 
m 



m 
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// Monitoring Schedules 

HRESULT NewMonitoringSchedule ( 

[in] OLECHAR name [MAX_SAGE_NAME] , 

[in] REFIID riid, 

[out, iid_is (riid) ] void** PPv) ; 

HRESULT EditMonitoringSchedule ( 

[in] DWORD schedID, 
[in] REFIID riid, 
[out, iid_is (riid) ] void** ppv) ; 

HRESULT DeleteMonitoringSchedule ( [in] DWORD schedID) ; 

// Holiday Schedules 

HRESULT NewHoliday Schedule ( 

[in] OLECHAR name [MAX_SAGE_NAME] , 

[in] REFIID riid, 

[out, iid_is (riid) ] void** PP V ) ; 

HRESULT EditHolidaySchedule( 

[in] DWORD schedID, 
[in] REFIID riid, 
[out, iid_is (riid) ] void** ppv) ; 

HRESULT DeleteHolidaySchedule ( [in] DWORD schedID); 



Table XXXIV 

IProfileserver Description 

moduleEdit Method 

NewMonitoringprofiieo Creates a new monitoring profile with the specified name and adds it 

to the database. The interface pointer for the monitoring profile is 
retrieved. 

EditMonitoringProfiieo R e t r j e ve an interface pointer on a specified monitoring profile, for 

direct manipulation of the monitoring profile. 
DeieteMonitoringprofiieo Deletes an existing monitoring profile. When calling this method, the 
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IProfileserver 
moduleEdit Method 



Description 



NewApplicationProf ile ( ) 

EditApplicationProf ile ( ) 
DeleteApplicationProf ile ( ) 
NewGroup ( ) 



EditGroup () 
DeleteGroup ( ) 

NewUser ( ) 

EditUser () 

DeleteUser () 

NewApp ( ) 
EditAppO 
DeleteApp ( ) 

GetAppIDByFingerPrint () 

NewProductVersionGroup { ) 
EditProductVersionGroup ( ) 
DeleteProductVersionGroup ( ) 
NewMonitoringSchedule ( ) 
EditMonitoringSchedule ( ) 
DeleteMonitoringSchedule ( ) 
NewHolidaySchedule ( ) 
EditHolidaySchedule ( ) 
DeleteHolidaySchedule ( ) 



caller must specify what is to be done with groups that are currently 
using the profile. If bReassignGroups is set to TRUE, then groups 
currently using this profile will be reassigned to the "Default" profile. 
Otherwise the deletion of the monitoring profile will be cascaded to 
any Groups using this profile 

Create a new application profile. The Application profile initially has 

no relationships to other objects in the profile database. The 

interface pointer for the application profile is retrieved. 

Retrieve an interface pointer on a specified application profile for 

direct manipulation of the application profile object. 

Delete an application profile. If any monitoring profiles are currently 

using this application profile, the relationship is deleted. 

Create a new group. Since groups are required to have an 

associated monitoring profile, an initial monitoring profile for the group 

must be specified. If the initial monitoring profile is specified as NULL, 

the group will use the "Default" monitoring profile. The interface 

pointer for the group is retrieved. 

Retrieve an interface pointer on a group for direct manipulation of the 
group object. 

Delete a Group. If bReassignUsers is TRUE, all users associated 
with the group will be re-assigned to the "Default" Group. If 
bReassignUsers is FALSE, all users associated with the group will be 
deleted. 

Create a new user. Since users are required to have an associated 
group, and initial group for the user can be specified. If the initial 
group is specified as NULL, then the user will be added to the 
"Default" group. The interface pointer for the user is retrieved. 

Retrieve an interface pointer on a user for direct manipulation of the 
user object. 

Delete a user. The relationship with the User's group, and an 
assigned monitoring profile (if one is assigned) will be deleted. 
Create a new Application. 
Edit an existing Application. 

Delete an Application. When an application is delete all recorded 

events and features for that app are also deleted. 

Get the ID of an Application using the unique Application Fingerprint 

String. 
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5 Report Database Object Interfaces 

Objects within the Report Database are exposed to clients of the DBOB as COM objects. The 
report database objects, and their interfaces are depicted in FIG. 53. 

IReport 

To access a specific report object for a report digest enumerated above, the following 

10 interface is used: 

interface IReport: IUnknown 

{ 

HRESULT GetDigest ( [out] TReportDigest* pDigest); 
HRESULT Persist ( [out] TReportDigest* pDigest); 
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HRESULT SetName ( [in] OLECHAR wszName [MAX_SAGE_NAME] ) ; 
HRESULT GetName ( [out] OLECHAR wszName [MAX SAGE NAME]); 



HRESULT SetCategory ( [in] DWORD dwCategorylD) ; 
20 HRESULT GetCategory ( [out] DWORD* pdwCategorylD) ; 

O 

sQ HRESULT GetCategoryName { [out] OLECHAR 

|U wszCategoryName [MAX_SAGE_NAME] ) ; 

25 HRESULT SetDescription ( 
fy [in] OLECHAR wszDescription [MAXJSAGEJDESCRIPTION] ) ; 

\i\ HRESULT GetDescription ( 

ffl [out] OLECHAR wszDescription [MAX_SAGE_DESCRIPTION] ) ; 

q 30 HRESULT SetReportFile ( [in] IStream* pStream); 

m HRESULT GetReportFile ( [out] IStream** pStream) ; 

is? ; 

'SB* 

pi // Report Parameters 

HRESULT EnumParams ( [out] IEnumParams** ppEnumParams) ; // r 

N 35 

HRESULT ModifyParam( 

[in] DWORD dwParamNumber , 

[in] OLECHAR szDe fault Value [MAX_REPORT_PARAM_LEN] , 
[in] OLECHAR s z Cap t i on [ MAX_RE PORT_P ARAM_LEN ] , 
40 [in] DWORD dwPropertylD, 

[in] DWORD dwPropertyPageNumber) ; 

// Report property Page Instances 
HRESULT EnumReportPropPages ( 
45 [out] IEnumReportPages** ppEnumReport Pages) ; 

HRESULT NewReportPropPage ( 

[in] DWORD pagelD, 

[in] OLECHAR wszCaption [MAX_SAGE_NAME] , 
50 [out] DWORD* pdwPageNumber) ; 
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HRESULT Modi fyReport Page ( 

[in] DWORD dwPageNumber , 

[in] OLECHAR wszCaption [MAX_SAGE_NAME] ) ; 



10 



HRESULT DeleteReportPagelnstance ( 

[in] DWORD dwPageNumber) / 



}; 



Table XXXV 
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IReport Method 

GetDigest ( ) 

Persist ( ) 
GetName ( ) 
SetName ( ) 
GetCategory () 
SetCategory () 
GetCategoryName ( ) 
GetDescription ( ) 
SetDescription ( ) 
GetReportFile ( ) 
SetReportFile ( ) 
EnumParams ( ) 
ModifyParamO 
EnumReportPropPages 
NewReportPropPage 
ModifyReportPage 
DeleteReportPagelnstanc 
e() 



Description 
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1 5 lEnumAppsinterface IEnumApps : IUnknown 
interface IEnumApps : IUnknown 

{ 

typedef struct tagAppDigest 
{ 

20 DWORD appID; 

OLECHAR 
OLECHAR 
OLECHAR 
BOOL 

25 } TAppDigest; 



appName [MAX_SAGE_NAME] ; 
appExeName [MAX_SAGE_NAME] ; 
appVersion [MAX_SAGE_APP_VERSION] 
appRe viewed; 
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[local] 

HRESULT Next( 



[in] ULONG celt, 

[out] TAppDigest *rgelt, 

[out] ULONG *pceltFetched) ; 



[call_as (Next) ] 
HRESULT RemoteNext ( 



[in] ULONG celt, 
[out, size_is (celt) , 
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length_is (*pceltFetched) ] 



}; 



TAppDigest *rgelt, 
[out] ULONG *pceltFetched) ; 

HRESULT Skip ([in] ULONG celt) ; 

HRESULT Reset () ; 

HRESULT Clone ([out] IEnumApps **ppenum) ; 

Table XXXVI 



IEnumApps Method Description 



NextO Retrieve the next AppDigest record from the enumeration. Returns 

the record celt id to be used to mark the seek for the next record. 

RemoteNext ( ) Retrieve the next AppDigest record from the remote server. This is 

mapped to Next() to support general retrieval of records from local 
and remote enumerations. 

skip ( ) Skip the next AppDigest record. 

Reset ( ) Reset the enumeration to the beginning. 

clone ( ) Copy the enumeration 



IEnumReports 

interface IEnumReports : IUnknown 

{ 

typedef struct tagReportDigest 
{ 

DWORD report ID ; 

OLECHAR reportName [MAX_SAGE_NAME] ; 

DWORD reportCategorylD ; 

OLECHAR reportDescription [MAX_SAGE_DESCRIPTION] ; 

} TReportDigest; 
[local] 

HRESULT Next( [in] ULONG celt, 

[out] TReportDigest *rgelt, 
[out] ULONG *pceltFetched) ; 

[call_as (Next) ] 

HRESULT RemoteNext ( [in] ULONG celt, 

[out, size_is (celt) , length_is (*pceltFetched) ] 
TReportDigest *rgelt , 
[out] ULONG *pceltFetched) ; 

HRESULT Skip ([in] ULONG celt) ; 

HRESULT Reset ( ) ; 

HRESULT Clone ([out] IEnumReports **ppenum) ; 

}; 

93 



Table XXXVII 



lEnumReports Method Description 



Next ( ) 



RemoteNext ( ) 



SkipO 
Reset () 
Clone ( ) 



Retrieve the next ReportDigest record from the enumeration. Returns 
the record celt id to be used to mark the seek for the next record. 

Retrieve the next ReportDigest record from the remote server. This is 

mapped to Next() to support general retrieval of records from local 

and remote enumerations. 

Skip the next ReportDigest record. 

Reset the enumeration to the beginning. 

Copy the enumeration 



IEnumCategories 

interface IEnumCategories : IUnknown 

{ 

10 typedef struct tagCategory 

{ 

DWORD category ID; 

OLECHAR categoryName [MAX_SAGE_NAME] ; 
□15 } TCategory; 
\f\ [local] 

Qg HRESULT Next( [in] ULONG celt, 

\\ [out] TCategory *rgelt, 

ffj 20 [out] ULONG *pceltFetched) ; 

a .. ; 

lire? 

pi [call_as (Next) ] 

g HRESULT RemoteNext ( [in] ULONG celt,. 

jU [out, size_is (celt) , length_is (*pceltFetched) ] 

25 TCategory *rgelt, 

i [out] ULONG *pceltFetched) ; 

ft; i 

~ HRESULT Skip ([in] ULONG celt) ; 

W 30 HRESULT Reset () ; 

HRESULT Clone ( [out] IEnumCategories **ppenum) ; 

Table XXXVIII 



}; 



IEnumCategories Description 
Method 
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Next ( ) 

RemoteNext ( ) 
SkipO 
Reset () 
Clone () 
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5 IDataManagement 

The definition of the server module 22 's IDataManagement interface is shown below: 
interface IDataManagement: IUnknown 

{ 

// Used to manage the active data management tasks 
10 // performed by the DBOB 

HRESULT GetTruncateModel ( [in] REFIID riid, 

[out, iid_is(riid) void** ppv); 
HRESULT GetCondenseModel ( [in] REFIID riid, 

[out, iid_is(riid) void** ppv); 

15 } 

The GetModel methods of IDataManagement allow the model dictating how and when data is 
condensed (archived) or truncated to be fetched and configured. The DBOB associated with a 
given server module 22 implements these interfaces but applies active data management only 
20 to data collected by the server's clients. In this way concurrency problems that arise when 
multiple threads attempt to groom the data are avoided. 



WW 
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Use of Iserver moduleAdmin 

When a Profile changes, the DBOB may be instructed to (optionally) notify the root 

Rj server module 22 installation in order to inform it of the users who have been affected by the 

hi 

jS 25 profile change. The root server module 22 can then instruct all other server module 22s in the 

system of the profile change. 
ffi In order to notify the root server module 22 of a Profile change, the DBOB will use 

p the Iserver moduleAdmin "GetSuperiorserver module() method of its corresponding server 

Q module 22 to navigate to the root server module 22. Once at the root server module 22, the 

30 DBOB will use the root server module 22's Iserver moduleAdmin: :RereadProfile() method to 
inform the server module 22 of the profile changes. It is then left to the root server module 22 
to inform all subordinate server module 22s of the change. 

Exceptions and Error Handling 

35 Table XXXIX describes the Database Object Broker responses to error results from 

the server implementation of the Iserver moduleAdmin interface. 



u 
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Table XXXIX 

Iserver moduleAdmin Error Response 

Method 

RepeatCommands ( ) E_FA1L - 

E_NOCOM - 
E_FAULT -] 

System Administration Subsystem 
System Administration: Client Side Components 

FIG. 54 highlights the client side components most directly associated with the 
System Administration Subsystem. Included among these components are the (i) Admin 
Console, used to administer the system of server modules and DBOBs, and (ii) Client 
Installation component, which installs the various client components on client host platforms. 
The later is implemented as an InstallShield program that installs the necessary software 
component packages on the client and configures the component for running on that client. 
Typically the InstallShield Setup.exe will be run once per machine to effect an installation. 
After the software has been installed management is performed using the Admin Console. 
The administration can however re-run Setup.exe at any time and reconfigure an existing 
installation. 

Admin Console 

The Admin Console is a tool typically used by system administrators. Through the 
Admin Console, the administrator sets up and configures the network of server modules that 
collectively make up the monitoring system. Through the Admin Console the administrator 
can determine whether a specific system is running, and if so, if it is running a client 
service/client monitoring agent. This feature allows the administrator to identify when the 
monitoring process has been circumvented, and also enables debugging of configuration 
problems. As was discussed above, a server module is aware of its subordinate servers and 
it's superior. With appropriate permission, the administrator may traverse the tree of server 
modules and perform administrative tasks at any node. 

Relationship Between Server Module and Admin Console 
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FIG. 55 depicts the relationship between the Admin Console and a server module 22. 
The Admin Console serves as the UI for the server module 22. There is typically not a one- 
to-one relationship between any set of Admin Console/server module instances, and with 
appropriate permission any Admin Console may connect to an arbitrary instance of server 
module 22. Through the admin console's use of the IServerAdmin interface implemented by 
server module 22, the administrator configures and maintains the server module 22 
installation. The Admin Console UI also provides access to features of the server module 22 
reserved exclusively for use by the administrator. Operations such as start/stop monitoring, 
client list, client ping etc. are performed directly, or facilitated by, the server module 22 to 
which the Admin Console is currently connected. 
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To ease the problems associated with administering a large network of servers, certain 
operations performed on a server via the Admin Console may optionally be repeated to all 
subordinates of the server to which the command was issued. The means to traverse the 
server tree, repeating a given command, is provided by the navigation methods of Iserver 
moduleAdmin. Such propagation of administrative commands through a server tree is 
illustratively represented by FIG. 56. 

interface Iserver moduleAdmin: IUnknown 



DWORD upSince; // a time_t 
DWORD numActiveClients ; 
BOOL repeat s Commands ; 
DWORD packet sQueued; 
DWORD packet sProcessed; 
DWORD dataltemsProcessed; 
DWORD dataTransmitErrors; 
DWORD packetQueueHighWater ; 

DWORD packetQueueHighWaterTime; // a time_t 
} Tserver moduleStatus; 

HRESULT EnumClients{ [out] IEnumString** ppEnumClient); 
HRESULT EnumActiveClients ( [out] IEnumString** ppEnumClient); 
HRESULT EnumSubordinateserver modules { [out] IEnumString** 
ppEnumserver module) ; 

HRESULT EnumActiveClientDigest { 

[out] IEnumActiveClientDigest** ppEnumClient) ; 
HRESULT RegisterAsSubordinate ( [in] LPOLESTR subordinate); 
HRESULT DeRegisterAsSubordinate ( [in] LPOLESTR subordinate); 
HRESULT GetSuperiorserver module ( [out] LPOLESTR* superior); 
HRESULT SetSuperiorserver module ( [in] LPOLESTR superior); 
HRESULT Getserver moduleStatus ( [out] Tserver moduleStatus* status); 
HRESULT RepeatCommands ( [in] boolean bRepeat ) ; 
HRESULT SetDBObjectBroker ( [in] LPOLESTR dbObj ectBrokerHost ) ; 
HRESULT GetDBObjectBroker ( [out] LPOLESTR* dbObj ectBrokerHost ) ; 
HRESULT GetDBOBAdmin ( [out] IDBOBAdmin** ppAdmin) ; 
HRESULT Syncserver moduleState (void) ; 

HRESULT GetEventLogEnumerator ( [in] LPOLESTR hostname, 



Referring again to FIG. 55, the use of ISageControlSink by the Admin Consolea 
allows commands to be repeated not only to each subordinate server but also to each of the 



typedef struct 



tagserver moduleStatus 



[in] TEvent Source source, 

[out] IEnumNTEventLog** enumEvents) ; 



} 
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server's clients. This simple feature allows a large system to be effectively managed and also 
permits management of sub-sections of the installation tree to be delegated. A command may 
be applied to the entire installed client base by traversing the server tree until a node is 
reached which has no superior (the master node), and by then applying the command to this 
master node and specifying that the command is to be repeated. To prevent widely applied 
commands from becoming a source of confusion, especially in the case of multiple 
administrators, commands travel down the server tree only. A server may optionally be 
configured to repeat only those commands originating from an instance of the Admin Console 
(an original command), and not those repeated by a superior server. 

Pinging Client Service to Determine Status 

Periodically, an administrator may wish to dynamically inquire about the status of the 
client service 50 on a remote client machine 12. The client ping feature of the Admin 
Console does not use an ICMP ping packet, which would verify only the presence of the 
target machine on the network. Instead, the Admin Console implements a ping which verifies 
not only that the client machine 12 is up and running but also that the client side monitoring 
software is installed and running as well. 

Referring to FIG. 57, the Admin Console pings a client service 50 by making all call 
to CoGetClassObject() for a Class ID (CLSID) which is present in the target machines 
registry only when the monitoring software is running. Recall that the client service 50 is 
typically not implemented as a COM object that can be activated, but the service is 
nonetheless COM-enabled. This means that the client service 50 CLSID is not found in the 
Windows registry, which precludes activation of the client service 50 via COM. When the 
client service 50 initiates operation, it makes a call to CoRegisterClassObject() in order to 
register its CLSID (implements only IClientService). The presence of this object indicates the 
presence of the monitoring software on the applicable client machine 12. If a remote process 
can successfully contact the client service 50 with CoGetClassObject(), then the result is a 
positive client ping. 
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Relationship to DBOB 

FIG. 58 depicts the relationship between the Admin Console and the DBOB 74. The 
IDBOB Admin interface is used by the Admin Console to control the active data management 
features of the DBOB (DBOB), and is fetched from the server module 22 through 
IserverAdmin::GetDBOBAdminO. Once the Admin Console obtains the IDBOB Admin 
interface pointer, it communicates with the DBOB. In a preferred implementation each server 
module 22 is associated with a DBOB and applies data management policies only to the 
DBMS to which that DBOB is assigned. This restriction not only simplifies the design but 
also allows for individual data management policies. 

UI Design 

Through the Admin Console UI, the administrator manages the server modules and 
their associated DBOBs. The administrative tasks are divided among the following functional 
areas: 

• Server Module Setup and Management 

• Database Housekeeping 

• Client Status (ping) 

• Client Monitoring 

In a preferred implementation the Admin Console UI has a single top level window 
divided into two panes (side by side). A tree view is presented on the left, which is always 
visible. The right hand pane is occupied by a tab box, which allows the user to choose 
between the functional areas identified above. The data presented in each tab is a function of 
the server to which the Admin Console is currently connected. 
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FIG. 59 shows a view of the Admin Console UI with the "server module" (default) tab 
selected. The user may navigate the server module 22 installation with the tree view. The 
relative view provided by the tree shows the SAGE system from the perspective of the current 
server (the server to which the Admin Console is currently connected). A relative view 
enables identification of the server module currently associated with the Admin Console, and 
reduces the amount of information presented in the UI to a manageable level. The elements 
of the tree view are interactive and may be expanded and contracted in the usual manner. The 
following mouse actions are available on elements of the tree view: 

• Double click on a server - The selected server is now the current server and all UI 
views update themselves. 

• Right mouse click - Posts a context menu containing the tab options appropriate for 
the object. 

• Left mouse click - Selects the object and adds it to an operation target list if one is 
visible on the current tab. 

Server Module Tab 

Referring to FIG. 60, selection of the server tab presents information descriptive of the 
server to which the Admin Console is currently connected. The view consists of: 

• Text fields specifying the superior server and the DBOB associated with the current 
server. 

• A list of operations which may be applied to the selected subordinate server(s) 

From within this tab the administrator may perform the following administrative operations: 

• Change the current server's superior. 

• Change the DBOB associated with the current server 

• Synchronize the server module with its clients 

• Force all clients to re-read their monitoring profiles. 

• Suspend data collection by the clients. 

• Resume data collection by the clients. 

• Flush client data to the server. 

• Flush client events to the server 

• Shutdown the selected server(s) and notify all clients. 

• Specify whether or not this server repeats command received by it's superior to its 
own subordinates. 

Whether or not operations are applied recursively to the subordinate servers of the current 
server is a function of the "Repeats Operations on Subordinates" check box. 



Data Management Dialog 
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FIG. 61 provides a screen shot of a Data Management dialog of the Admin Console 
user interface. This interface dialog is primarily used to control the data management features 
of the DBOB associated with the current server. As an example, this dialog can be used to 
control the automated "push" of monitoring data from the collection schema to the reporting 
schema. 

Client Tab 

FIG. 62 provides a screen shot of a "Client" tab of the Admin Console user interface. 
The user interface presented upon selection of this tab is similar to that presented upon 
selection of the "Current server" tab. Selection of this Client tab enables performance of the 
following operations: 

• Force all clients to re-read their monitoring profiles. 

• Suspend data collection by the clients. 

• Resume data collection by the clients. 

• Flush client data to the server. 

• Flush client events to the server 

• Ping a historic or currently active client. 

Event Log Tab 

FIG. 63 provides a screen shot of a "Log" tab for the Admin Console user interface. 
As is indicated by FIG. 63, important transactions and errors reported to the NT event log by 
the current server (or its clients) are presented and ordered by timestamp. Significant errors 
catalogued include: 

• Client connection failure. 

• Corrupt data sent from client. 

• Unable to access database. 

• Error in database transaction. 

• Out of memory error. 

Status information such as commands repeated by the applicable server module, as well as re- 
start information and the like, may also be shown. The log depicted in FIG. 63 may serve as 
both as an audit trail and a debugging tool. 
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Monitoring User Interaction with the World Wide Web 
Overview 

A primary goal of the web-based monitoring system of the present invention, 
hereinafter "NetEchoes™", is to produce a system capable of monitoring user interaction with 
the world wide web. While the implementation of NetEchoes™ may be different from the 
implementation described above, a similar level of monitoring detail may be effected in each 
case. In comparison with the server log file analysis tools widely used today, both 
NetEchoes™ and other implementations of the present invention facilitate collection of data 
in greater detail and production of entirely new categories of data metrics. 

By implementing web-based monitoring in a language used to construct web-delivered 
applications, the monitoring system itself can be embedded in any web page. This negates the 
need for a separate installation step on the client and allows the sample of users monitored to 
grow almost without bound. Whenever a user loads a page containing the embedded 
monitoring, the browser responds by initializing the monitoring system (subject to the security 
settings of the browser). 

The producer's web pages are modified at the server to include a JavaScript that 
gathers the events and desired measures while the web page is active within the browser. 
During the time the page is active and when it is unloaded, the JavaScript sends information 
about the users behavior and experience with the web page being viewed back to a data 
collection web server farm. JavaScript is preferably used as the web-centric language in 
which to implement the monitoring for the following reasons: 

• It can be embedded in the page itself in text form causing little to no impact on the 
download time of the page. 

• It has built in features to access all of the components of a web page and interact 
with them. This concept is discussed in the section "Document Object Model 
(DOM)". 

• Unlike an ActiveX control or Netscape plugin, JavaScript has no ability to interact 
with the user's desktop or file system assuring users that no malicious actions will 
be performed. 
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The monitoring of web pages via an embedded JavaScript module is extremely 
valuable for e-commerce and e-business web sites. It allows for instant (realtime) feedback 
on how visitors are interacting with the site. 

Questions such as: 

10 • How long did the page take to download (from the client's perspective)? 

• What was the previous page that led the user to this page? 

• How was the user fed to the site? Was it fed by a search engine, link from within 
an email, advertisement on another site, link from another site, or bookmark or 
type in? 

15 • How did each visitor behave on the site? Did they make purchases, sign up for 

newsletters, or behave in other desirable ways? 

• How did visitors fed from different sources behave differently? Which referral 
source fed the most visitors who purchased, subscribed or behaved in other 

□ desirable ways. 

y3 20 • What is the visitor breakdown for the site. How many visitors are returning or 

In new? Of those returning, how many have behaved in desirable ways in the past 

^ (i.e. have they purchased, subscribed, or taken another action)? 

j\ • What advertisements did the user actually view (ad impressions)? 

J fj • Which elements (images and ads) on the page were the slowest to load? 

25 • What were the most commonly clicked on links of page? 

• How may pages on the site were viewed previously to this one? 

• How long was the user active on the page? 

P • What advertisements were clicked-through on the page? 

=3 • Did the user hesitate during any part of a process or task? 

;J 30 • Was any operation, task or form abandoned? 

5 • When a form was abandoned, what was the last field filled in by the user? 

• How the user left the site (advertisement selection, other URL, typed text in the 
GOTO box, etc.)? 

• Where did the user go when they left the site? 

35 • How long did each page take to load from the user's perspective? 

• How many of the pages the user viewed came from cache vs. a load from the site? 

• What path did the user actually take through the site (modified to include cached 
pages)? 

• Did the user interrupt the loading of the page by pressing the stop button, clicking 
40 on a link, etc.? 

• Did the user leave the site immediately after interrupting the download of a page 
(interrupt abandons)? 

• What browser and platform (operating system) was the user using? 

• Did any errors occur while the user was interacting with the page? 
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• Did the user make any purchases while on the site? 

could easily be answered. There is no way to get this level of detailed information from the 
logs generated by the web server. 

This monitoring technology could also be applied to a sample of users who have 
intentionally elected to permit the monitoring system to monitor all web activity. The data 
collected could be used to produce ratings of web sites similar to the Nielsen system currently 
applied to select television viewers. This data would not be anonymous. The demographics 
of the users would be well known. This type of information is very valuable to advertisers. 
An internet service where advertisers could enter a description of their target audience and 
receive a list of the top ten recommended sites to place their add would likely be an enormous 
success. 

User Monitoring vs. server module Monitoring 

There are two general sources of web usage data; user data and server log file data. 
User data contains very detailed information regarding every site the user has visited and the 
type of actions performed by the user at those sites. Server module logs contain very granular 
information regarding activity of all users of a single site. 

Many products exist to perform analysis of server log file data. The data is generated 
automatically by the server making it essentially free and the tools themselves are relatively 
inexpensive. The information in the logs is relatively easy to interpret and provides valuable 
information to the operator of a web site. For example, a web site operator can use server log 
data to: 

• Assess the popularity of a site in comparison to other like sites 

• Determine the most viewed information offered by the site 

• Find the most popular paths through the site (generalized) 

• Identify the areas on the site from which users most often leave for other sites. 

An example of the contents of an HTTP server log is shown below. Each entry shows 
various information such as the requestor, the time, the protocol operation (GET, HEAD, 
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5 POST, etc.) the page requested and information regarding the browser which issued the 
request. 

206.132.57.231 - - [04/Jun/1999:09:24:17 -0700] "GET /images/nav/elevatorO jpg HTTP/1. 1" 200 4873 
,, http://www.limesoft.com/left.htmr' "Mozilla/4.0 (compatible;MSIE 5.0; Windows 98)" 
1 0 206.132.57.231 - - [04/Jun/1999:09:24:17 -0700] "GET /images/nav/elevator2 jpg HTTP/1. 1" 200 5408 

"http://wwJimesoftxoirtfleft.htmr "Mozilla/4.0 (compatible;MSIE 5.0; Windows 98)" 
206.132.57.231 - - [04/Jun/1999:09:24:17 -0700] "GET /images/nav/elevatorl jpg HTTP/1. 1" 200 5526 
"http://www.limesoft.com/left.html" "Mozilla/4.0 (compatible ;MSIE 5.0; Windows 98)" 

15 The server log contains a great deal of data about what requests (hits) were made of 

the web server, but does not contain any data about the users' experiences with the pages 
viewed. Analysis of the data can yield the answers to many important questions about the 
performance and page volume handled by a server but it can never provide much information 
about the users interaction with each page. 

20 Data collection performed at the client within the browser yields all of the server- 

O 

s£s collected information, but can also yield a much richer stream of data that includes details of 

S 

&i the user's interaction with a page. These details serve to fill in the gaps in the information 
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offered by a server log and can be more accurate. One reason client side monitoring is more 
accurate is the ability of web browsers to "cache" pages so that a second, or subsequent visit 

Q 

m 25 to a page does not generate a "hit" to the remote server, but rather the browser simply pulls 

Q 

m the page from its local disk cache. There are actually two kinds of page caching, caching by 

the browser and caching by a proxy server. The goal of both schemes is to improve 
performance from the perspective of the user. Caching has the effect of hiding page load 
activity from the server. Recent figures suggest that server log file analysis fails to show as 
30 much as 20 to 40 percent of a sites real traffic. Many ISPs such as America Online (AOL) 
operate massive caching schemes in order to deliver higher performance to their subscribers. 
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5 For many web site, tracking page view data is essential. Many sites generate revenue 

on advertising banner "impressions." More accurate page view data tracking both cached and 
non-cached pages may result in additional revenue. 

If an ad is placed at the bottom of a page there is a high likelihood that it will not be 
initially visible when the page is loaded and many users will not bother to scroll down to 
10 expose it. Data which could prove that an ad was actually viewed would be help the seller of 
ad space price their page "real estate" appropriately and assure a buyer that their money is not 
being wasted. 

Web Documents 

The producer of Web content develops a site using combinations of HTML, 
15 JavaScript, and embedded objects such as applets, Active-X controls, and plug-ins. The 
development of sites has continued to progress and advance with introduction of new 
technologies and techniques. These technologies on the client side have been packaged into 
what is now called Dynamic HTML or DHTML. 

The following briefly defines the technologies used to build web content and their 
rf: 20 utilization in connection with NetEchoes . 



Hyper Text Markup Language 

Hyper Text Markup Language or HTML provides hypertext links that allow the user 
to move to different pages. Originally, HTML allowed combinations of rich text and images 
25 to be viewed with a Web Browser with the content being retrieved from locations anywhere 
on the Internet, forming a World Wide Web. HTML has since progressed to provide a number 
of different ways that content can be presented. A plethora of new mark up tags have been 
introduced that control not only the content but also every aspect of content; presentation, 
location, indentation, font type, font size, font style, etc. 



30 Besides anchors, two aspects of HTML presentation may be particularly useful in the 

context of NetEchoes™ - frames and forms. Frames allow content from different locations to 
be presented at the same time. Various issues surrounding the implementation of frames are 
described below. 
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Forms allow users to provide information via input elements such as buttons and text 
fields capable of being tracked by the web-based monitoring system. These are the items that 
must be observed and measured, and depending upon the browser, are presented as different 
window objects. 



JavaScript can be included in an HTML document using the <SCRIPT> tag. It can 
perform functions on the client side without having to connect with the server. For example, 
forms can be verified to contain the correct entries before they are submitted to the server. 
Also with JavaScript, the page can respond to the user actions, such as changing the button 
image when the mouse pointer moves over the button, or the button is pressed. 

JavaScript acts on objects that make up a Document Object Model The DOM 
identifies what objects within a document can be observed by a script and what actions can be 
performed on these objects. Therefore, the actions that can be performed using JavaScript 
depend greatly on the DOM. 



Applets are written in a platform independent language - Java. In particular, they can 
be included within the content on an HTML document without user interaction and in most 
cases without user notification. Applets operate with security restrictions that limit their 
scope to an HTML page and limits their communications to the originating server of the page. 
With JavaScript, using Netscape's Live Wire or Microsoft's Active-X technologies, scripts can 
interact with applets and visa versa. Therefore the scope of the applet has been extended to 
cover the document in which it is embedded and even allows applets within the same page to 
communicate. 

Like other embedded objects, the challenge to NetEchoes™ is observing and tracking 
the activity the takes place within the applet. The implementation of Java's Graphical User 
Interface objects can be different depending upon the operating system and browser. 

If the applets have been developed using Netscape's Live Wire or Microsoft's Active-X 
enabled JDKs, then the applet is open to full interaction with JavaScript. In fact, Java 



JavaScript 



Applets 



108 



packages can be used to extend JavaScript functionality. Therefore Java and Applets become 
an enabling technologies for embedding NetEchoes™ functions within HTML documents. 

Plug-ins and Active-X Controls 

Like applets, Netscape Plug-ins and Microsoft Active-X Controls, are embedded 
objects within an HTML document that allow the user to extend the functionality of the 
document. Unlike applets, these components must be installed on the client prior to their 
activation within a Web page. They require user consent and action. Active-X controls can 
more or less be installed automatically after the user has given consent, and under certain 
circumstances, the user's consent is not necessary. Netscape plug-ins usually require restart of 
the browser before they can be used, and require considerably more interaction by the user. 

Once installed, the plug-in or Active-X control is activated by an <EMBED> or 
<OBJECT> tag within the document, typically to handle a MIME type not handled by the 
browser itself. The embedded object is deactivated when the user moves to another page. 
Once activated, they have full run of the computer system, i.e. there is no security context 
around the object like that of a Java applet. 

Like applets, Plug-ins and Active-X controls are open to full interaction with 
JavaScript and are enabling technologies for NetEchoes™. 

Dynamic HTML 

Dynamic HTML (DHTML) is an attempt to extend the document object model and 
separate once again style from content markup through Cascading Style Sheets (CSS). The 
DOM standardizes and extends the number of objects that can be observed and manipulated 
by a script, such as Java Script, JScript, even Perl. 

CSS greatly extends the control of style aspects of presenting the content, while 
removing a number of tag attributes in current HTML. The cascading refers to the context of a 
style sheet - embedded specific to an instance, global effecting the entire document, user 
preference defined by a user, etc. 
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DHTML extends the effect of JavaScript even further, hence extending what 
NetEchoes™ can observe. On the other hand, it still developing like the original HTML with 
different DOMs and capabilities for both Netscape and Microsoft browsers. There is however 
a possibility to implement DHTML documents the operate across both models or allow the 
same functionality with different implementations 

Document Object Model (DOM) 

All scripts embedded within an HTML document operate upon a Document Object 
Model (DOM). The W3C is attempting to develop a standard model with both Netscape and 
Microsoft. This standard DOM would allow scripting languages to be embedded in HTML 
documents and have access to the same document objects. Currently each browser exposes 
both window and document objects that can vary from one browser to the next. These objects 
are referred to as the Web Browser Object Model (WBOM). 

Because both Netscape and Microsoft support JavaScript there is a common set of 
objects accessible in both browsers. As the Browser Wars have moved into Dynamic HTML, 
there has been a divergence in the events and objects observable within each browser. For the 
purpose of this document we will concentrate on the common Web BOM for JavaScript. 

The JavaScript objects are listed in Table XXXX. Associated with each object are 
properties or attributes that can be accessed and events that can be observed for each object. 
The properties of browser object can be accessed by simply including a script within the 
HTML or via a Java Applet or Active-X Control. To capture the events requires adding 
specific attributes to HTML tags to tell the browser what JavaScript function should be called. 
Event handling within the DOM is discussed in detail in the following section. 
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Table XXXX 
JavaScript Web Object Model 



Web#j|et 



08 

3 s s 

CP 

Q 
CP 
□ 

Q 



I window 



The actual content area of the browser displaying the current document. 



|l6cati|?na^ ^B|,|gli 



history 



link, area 



I A list of previous pages visited during this browser session. 



Hi! 



»*— — <-*r~ 



TM|roo^)£the ; tref!Qf@jg^ lfiat7m^^m|thei;coritent of ffie^ihdo^S^ 



The <A> or <AREA> tag object within a document containing a HREF 
attribute. 





[The <A> ta g; object within a document cqnt 




form 

t ■ 


[The <FORM> object within a dbcumer 
| elements. V. . : : ■! " ., 


t|;^^^^^i^^^^^^^^ 


image 


; The <IMG> object within a document. 




• frame ^ff^fcil?'? * f ' 


>M^^W' 7V: - 'V - ' ■ ■ : : ,; ."- ■ ■ 
:MpiIdSw object contained! within a docum 

U-^i^a^aS™.^^.^ V ^ ; - 




pppleitsM;r- 5 ; -plugins, 


jThe <APPLET>, <EMBED>, and <01 
[documen L _ _ 


IM,^flftMbjel^*wiffiin»i 



mimeTypes 
options 



1 1 The list of helper applications and plug-ins to handle each mime type. 
The <OPTION> tag objects within a document. 



The insertion of attributes to capture events for NetEchoes™ is referred to as instrumenting 
10 the HTML. Before addressing instrumentation the objects that populate the DOM are 
presented. The following summarizes each object and the attributes and events associated 
with each object. 



Window Object 

15 The Window object represents the object itself and in particular, the content area of the 
document. From the highest or Top window, the browser state can be determined, and some 
aspects of the browser controlled. For example, new windows can be opened and the 
scrollbars controlled. For frames, the window object provides links to the parent window and 
to sibling windows. 

Ill 



Table XXXXI 





Window Object Properties 

l^^^S ^ ^ 

BobleaEn : Specifies whether a window has been closed (Boolean) 

' 'jjjf ' 

Browser]; 

The number of frames in the current window 

iThenamefof a window 1 * 





integer 



length- 

opener ; ^; 1|; \ string ; ■ The window that opened this- window -or fram : ; 

self or top object Refers to the current window ' lxl : M;14, - -^W^ 



,vst^ti^pp;^ps tstarik * ' The contents of the Web browser's status line. ' ' 



Table XXXXII 

Window Object Events associated with the <BODY> or <FRAMESET> tags of the document. 



^^^g^^^mm^^ - \ ■■■ ^Q grrQr occurs ' in the "window " *" ' * ' " * " *" 



oriUhload 



The focus is applied to the window 



onFoeus: WWM 

KioS^^ loading a document into the window " 



The u^er exits from the docilmSirt within the window. 



Though windows can be created and their content scrolled, the ability to know what 
content is being display is curiously missing. DHTML does provide extensions in the window 
object allowing measurement of mouse position within an HTML page and the size of the 
viewing area. 

The event model has been expanded in both Netscape 4.0 and Internet Explorer 4.0 to 
measure mouse events and keystroke events relative to targets objects within the page. Thus 
window events in general can be monitored from <BODY> or <FRAMESET> tags without 
having to instrument all tags within an HTML page. This greatly simplifies the 
instrumentation having only to append attributes to the <BODY> or <FRAMESET> tags. An 
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example of this instrumentation technique is presented in the section entitled "Instrumentation 
of Web Documents". 



Location Object 



Table XXXXIII 

Location Object Properties 



Descri ption , 



Properties i Type _ 

href ! string ^ \ The entire URI^ including all the subparts. 



string 



\ The hostname and port number. 



integer ! fThe port number of the URL • 

MS 

; : Aiiy CGI arguments after the first # in the URL 



in 



j name v" ; string The name attribute of the anchor. 
\ assign(string) j Method \ Set the href to the value you specify. 



The properties of a location object can be changed as well as observed. This provides 
a mechanism for navigating via JavaScript as well as passing content back to the 
server. 



History Object 



Table XXXXIV 

History Object Properties and Methods 



K^rtiei#illpBllg^ 

current -string Contains the URL of the current history entry 



\ previous string Contains the URL of the previous history stack entry 



m^r^ 



m 



backO Method Goes back one entry in the history list 

: go(num) Method Goes forward num entries in the history stack if num > 0 

otherwise it goes backward -num entries. 




SMetKoIlftl 
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5 The history is a protected or trusted object in Netscape requiring signed scripts and 

HTML to allow permissions to be granted by the client. With Universal Browser Read 
privilege, the script can access the entire list of sites visited during this session. With 
Universal Browser Mail privilege, used to generate a form letter to filled and sent from the 
client, the script can access the email address. 

10 

Document Object 

The document object is the central or root object for accessing all the objects within an 
HTML page. Associated with the document are a number of methods that allow content to 
1 5 dynamically written to the browser. 
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Table XXXXV 

Document Object Properties 




The URL of the document from which this document was called 



URL 


string 


The complete URL of the document 


VlinkColor 


string 


The color of the document's visited links 


closeQ 


metho 
d 


Closes the specified document 


evalQ 


metho 
d 


Evaluates string as JavaScript code and returns the result. 


; openO 


metho 


Opens a stream for a new document. This document is meant to 




d 


be filled with the output of calls to the document. write() method. 


write(expression) 


metho 
d 


Writes one or more HTML expressions to the document. 
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Writeln(expressio metho Identical to writeQ except appends a newline. 

n) ■. . . d ..' ;..,i,;^,:„,.;- ; ; \/ : '[ ". .. ' \\';/ : ^ : 

Typically attributes are read only. The document's location is read only. However the 
window's location, which has essentially the same information, can be set causing the 
browser to load the new URL. 



Table XXXXVI 

Document Object Arrays 



anchors 



; arguments 

embeds 

111 
\ forms 

history 

[images 

links 

I mimeTypes 

! options 
; plugins 



One for each of the <A> tags containing the NAME attribute 



One for each argument of a JavaScript function 



One for each <EMBED> tag "■ 

i mIIiB 

Refers to the current window 

One for each entry injhe history list for a window 

One for each of Me <IMG> tag ; v ^ 

One for each of the <A> or <AREA> tags containing the HREF attribute 
One for each MIME type supported by the client Web browser, its helper 
applications, plujg-ihs, and (for Internet Explorer) ActiveX Controls. P ; 
One for each of the <0^ : r -'c^-r , T; 

i One for each plug-in installed on the client Web browser 



Note that in the list above, one gets a fairly complete view of the document including 
all tags or elements if necessary. Accordingly, the NetEchoes™ implementation does not 
require objects to be probed for using Microsoft Active Accessibility (MSAA). 



Link, Area, Anchor Objects 



Table XXXXVII 

Link, Area, Anchor Object Properties 



href string 



The entire URL, including all the subparts. 
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host string The hostname and port number. 



, port integer The port number of the URL 



hash string Any CGI arguments after the first # in the URL 

name string The name attribute of the anchor. 

assign(string) Method Set the href to the value you specify. 

Table XXXXVIII 

Link, Area, Anchor Object Events associated with <A> or <AREA> tags 



onClick •:. A link object is clicked 

onMouseOut | The mouse passes out of a link or area object 



Note that the Link and Area objects are implemented as Location Objects allowing 
links to be manipulated. 

Form Object 

The form object is important since it represents, aside from the link, the most user 
action intensive aspect of an HTML page. From the Form object the user entries for each 
element and the submit and reset buttons are observed. 



Table XXXXIX 

Form Object Properties 



name 



string 



The value of the form's NAME attribute 



string . The value of the form's ACTION attribute 



^method 



action 



— .. .7 ' V.J 




The number of elements in the form 
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target 



string Window targeted after submit for the form response 



reset() 
submitQ 



Method Reset all the elements within the form. 
Method Submit the form. 



Table XXXXX 

Form Object Events associated with <FORM> tag 

i onreset : The <RESET> button is clicked or the resetQ method is called 

The form elements are listed with the form object and the following gives the properties and 
events that can be observed for each element. 



Table XXXXXI 

Form Element Properties 



name 



: string The entire URL, including all the subparts. 



defaultValue i t string : The hostname and port number. 



focusQ 



; Method : Moves the input focus to the specified object 



selectQ 



j Method Selects the specified object 



fMethod^ 



Isiiteiitp ■ \.f f 

: 




Table XXXXXII 

Form Element Events 




The user moves the input focus to the field, either via the Tab or a mouse 
click. 



jmmmm 


llfte u^ out §|{this field ? • . 


mmmmsmx 


|ftSelectH,^fl 






w*mmmi 
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Image Object 



Table XXXXXIII 

Image Object Properties 



border 



; string : The value of the image^s^ORJDER attribute 



r height D integer The value of the image's HEIGHT attribute 



lowsrc 



! integer -., \ The value of the image's LOWSRC attribute 



src string The value of the image's SRC attribute - :u- • 



wmr ^ * - ; "" 



width 



Name 



integer The value of the image's WIDTH attribute 



Table XXXXXIV 

Image Object Events associated with <IMG> tag 



onAbort . 1 The user aborts the loading of an image, such as by clicking the Stop button 



onLoad 



An image is completely loaded 



HTML Elements 

With DHTML, every HTML element and its attributes can be accessed. What can be 
done with these objects varies between Netscape4 and IE4. In both cases, the Style using 
<STYLE> tags can be applied to these objects to effect any aspect of the element style - 
including fonts-type, font-size, borders, padding spaces, etc. JavaScript supporting the 
generating of the <STYLE> tags on the run and also changing the presentation of the element 
during runtime. 
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5 Embedded Objects 

Applets, plug-ins and ActiveX Controls as embedded objects are accessible via the 
Browser Object Model. As can be seen in Table XXXXVI, these objects can be accessed via 
the document object. With JavaScript the methods of the embedded object can be accessed 
directly. In fact, the JavaScript functionality can be extended by direct calls to singleton 

10 methods within a package, such as the system object, or the methods of the instance of an 
applet. 

Likewise, Java applets and plug-ins that have Java APIs can have full access to the 
DOM and JavaScript functions. This is done by importing the Netscape package that defines 
the JSObject and JSExcepton classes. ActiveX Controls operate as extensions of the browser 
15 window and are directly integrated into the browser such that JavaScript calls are direct via 
the name for the embedded object and likewise the control has direct access to the WBOM 

0 and JavaScript. 

IH In a cooperative environment, where the producer desires specific measures to be 

K\ performed, the embedded objects that must be monitored can be instrumented to provide the 

!H 20 measures such that the monitoring and measurement can be integrated via JavaScript and 

yy 

01 other embedded controls. 

S3 

CH For example, suppose a producer wants to measure whether or not specific places 

O 

Hj within his web site are viewed by a customer. By inserting a "cooperative" applet that reports 

:J 25 number of paints, one can determine whether an area is exposed and for how long, and by 
measuring mouseovers, one can determine if the user even anticipated following a link. For 
Java applets, additional instrumentation can be added to measure activity within the applet 
either through instrumented packages for observables, an "accessibility" API, or even 
"wrapping" existing Java objects. 

30 

If the producer wishes to encourage and extend monitoring of its content, a more 
direct means of measurement can be incorporated. If the presentation of the content depends 
upon popular plug-ins or ActiveX Controls that have already been installed by the customer, 
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5 then via JavaScript and Java wrappers, the functions of these embedded controls can be 
monitored as well. 

NetEchoes™ Overview 

FIG. 64 depicts the major system components of NetEchoes™ 100 along with the 
10 context in which they appear. The components within the HTML Page 104 are known as the 
NetEchoes™ Client components and interface with the NetEchoes™ server module 1 10. This 
diagram also depicts steps that occur in the communication between Producer Web server 
module 1 14, Browser 118 and NetEchoes™ server module 110. 

1 5 Steps involved in Producer / Browser / NetEchoes™ server module relationship: 

1. Producer Web server module Serves HTML Page: The producer web server 
114 serves up HTML Page 104 (i.e., an HTML document) in response to a 
request from Browser 118. The HTML Page 104 incorporates an embedded 
p NetEchoes™ Tag 120. 

y3 20 2. Browser loads and renders the HTML Page: When the browser 118 loads and 

J renders the HTML Page 104, the effect of the NetEchoes™ Tag 120 is to 

W request the Instrumentation Script 124 from the NetEchoes™ server module 

3. NetEchoes™ Tag References the Instrumentation Script: The NetEchoes™ 

25 Tag 120 instructs the browser 1 18 to load the Instrumentation Script 124 from 

^ ' the NetEchoes™ server module 1 1 0 and begin execution. 

^ 4. The NetEchoes™ server module 110 serves up the Instrumentation Script 124 

m in response to the request from browser 118. 

O 5. The Instrumentation Script 124 executes on the HTML Page 104 and sends 

fy 30 collected data back to the NetEchoes™ server module 1 1 0. 
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NetEchoes™ Client 

This section describes the technology employed on a NetEchoes™ client within and 
35 HTML page loaded by a browser. 

Event Handling within the DOM 

JavaScript programs use the event handling feature of the DOM to activate web pages 

and build web apps that respond to user input. Like the DOM itself, the event handling model 

40 of the DOM varies significantly between JavaScript implementations, however there is a set 

of least common denominator events that are similar across all JavaScript implementations. 
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In the previous sections tables were presented identifying event handlers like 
onmouseover as properties of an object of the DOM. These properties may be assigned by a 
script or they may be expressed as the values of HTML attributes belonging to an object in the 
DOM. For example, the onLoad event handler can be specified with the onLoad attribute of 
the <BODY> tag and the onReset event handler can be specified with the onReset attribute of 
the <FORM> tag. This synergy between the HTML markup language and JavaScript reflect 
the integration of scripting with the DOM. The value of an event handler may be set to an 
arbitrary string of JavaScript code but typically a function is defined and the handler assigned 
to the function definition. 

Table XXXXXV presents the least common denominator events that exist between the 
Netscape and IE implementations of JavaScript. The third column indicates the object or 
objects which support the event type. 



\<; TABLEXXXXXV _ ^ 





Loading Interrupted : W 


: Image : ■ . ; : ; : : - : :1||||{f ; ; f 


ohBlur 


Element loses input focus 


Text Elements, Window, Ml other 
elements ; / ! \}. ■ 


onChange "W 


User selects: or deselects an item or 
enters text and moves input focus to 
another element - : . 


■ Select, text input elements 1 : '' ;; ??|f 


onClick • 


User clicks once. Return false to cancel 
default action (i.e., follow link, reset, 
submit) ■ ,<.•••.• 


Link, Button elements . 


li^iPj^iii 


User clicks twice. ''^IIIH 


Document, Link, 4 Image, button 
elements 


onError 


Error occurs while loading an image. 


Image ■ 


onFocus 


Element given input focus. 


Text elements, Window, all other form 
elements 


onKeyDown 


Key pressed by user. Return false to 
cancel. . >. : "Mi 


Document, Image, Link, text elements 


onKeyPress 


Key pressed by user; a combination of 
onKeyDown and onKeyUp. Return false 
to cancel. S 


Document, Image, Link, text elements 


onKeyUp 


Key released by user. 


Document, Image, Link, text elements 


onLoad 


Document or image finishes loading. 


Window, Image 


onMouseDown 


User presses mouse button. Return false 


Document, Image, Link, button 
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to cancel. 


elements 


onMouseOut 


Mouse moves off element. 


Link, Image, Layer 


onMouseOver 


Mouse moves over element. For links 
return true to prevent URL from 
appearing in the status bar. 


Link, Image, Layer 


onMouseUp 


User releases mouse button. Return 
false to cancel. ^ 


Document, Image, Link, button 
elements 


onReset 


Form reset requested. Return false to 
prevent reset. 


Form 


onResize 


Window is resized. > • 


Window ,.; - V:V ..: 


onSubmit 


Form submission requested. Return 
false to prevent submission 


Form . . 


onUnload 


Document is unloaded. 


Window 



In addition to the event handlers discussed above a more generalized event processing 
exists in both the IE and Netscape versions of JavaScript. Both Netscape and IE have the 
concept of an extension to the core client-side event model commonly referred to as the 
"fourth generation" event model. This name refers to the appearance of an Event object in IE 
4 and Navigator 4. 

Netscape and IE implement the concept of the event object with distinct differences 
and as yet no clear standard has emerged, however the two models can be reconciled easily 
with the addition of JavaScript code that determines the browser platform and reacts 
accordingly. 

Table XXXXXVI outlines the properties of the Event object and the differences in the 
object's properties between IE and Netscape. 

Table XXXXXVI 



Event type 



type property, a string that contains the 
event type name. 



type property, a string that contains the 
event type name. 



srcElement property 
button property: 1, 2 or 3. 



Event source 



target property. 



Mouse button 



whith property: 1, 2 or 3 



Key 



which property holds Unicode character 
code (not a string) 



keyCode property holds Unicode 
character code (not a string). 
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Modifier keys 


modifiers property is a bitmask 

containing the Event.ALT MASK, 

EventCONTROL_MASK, 

Pvpnt MFTA MA SfC and 

Event. SHIFT^MASK flags. 


altKey, ctrlKey, and shiftKey properties 
contain individual boolean values. 


Mouse position 


pageX and pageY properties specify the 
coordinates relative to the web page; 
screenX, screenY specify coordinates 
relative to the screen. ; i 


clientX and clientY properties specify 
coordinates relative to the web page; 
screenX screenY specify coordinates 
relative to the screen. 



Access to the Event object in Netscape is via an argument passed to the event handler. 
In IE the Event object is available via the global variable event. 



As mentioned previously, reconciling these two different event models is not difficult; 
10 it requires some code to detect the browser and take different action depending on it. An 
example of an event handler which functions equally well in both the Netscape and IE 
environments is show below: 



m 



function myHandler(e) 

15 { 

if (navigator. appName . indexOf ("Microsoft") != -1) 
e = window . event ; 

// common processing using the local variable e to reference 
20 //the Event object 

} 

The last major difference between the Netscape and IE event handling models is how 
events propagate through the hierarchy of objects occupying the DOM. Netscape uses an 
25 event "capture" model while IE uses an event "bubbling" scheme. 
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The Netscape model introduces new methods on the Window and Document objects 
to allow specified events to be captured, giving the script a chance to process an event before 
the object which actually generated it. When handled, the event can be inspected and then 
either discarded, forwarded to the source object, or rerouted to another object or handler. 
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Because IE 4 includes all HTML elements in its DOM event handlers exists on all the 
elements. The presence of consistent event handlers on all objects in the DOM allows events 
to be generated and then propagate up through all containing HTML elements to the 
Document and Window object. At any point during this process the propagation of the event 
through the DOM can be halted by setting the Event's cancelBubble property. 

Instrumentation 

The richness of the JavaScript event model and the synergy between JavaScript and 
the objects occupying the DOM, allows for the creation of a script which can interrogate the 
DOM for its constituents and then set handlers which provide feedback on the user's 
interaction with the document. This process is referred to as instrumentation. 

All of the properties of any element within a page are available and can be modified 
regardless of how the page was generated or customized. This means that the instrumentation 
can be performed during and after download on the client. No server instrumentation is 
required except the incorporation of the NetEchoes™ script in the page somewhere in the 
<BODY> of the document to be instrumented. 

NetEchoes™ "Tag" 

An HTML document is instrumented by the placement of a <SCRIPT> tag (indicating 
the presence of a JavaScript) within the body of the document. One of the important aspects 
of this tag is that the JavaScript referenced can be delivered from an alternate web server - it 
need not be the one that delivered the other document content. The HTML fragment below 
illustrates the general pattern of the tag and its placement within the document: 

<BODY BGCOLOR=blanchedalmond topMargin=50> 

<SCRIPT SRC="http: //stats .klsoft .com/script / Instrumentation. js"> 
</SCRIPT> 

In this fashion, the script is delivered from an alternate web server (distinct from the 
content server), potentially by a partner the producer works with. 
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5 Instrumentation of SSL-Delivered Pages and NetEchoes™ "Smart Tag" 

Most commonly used web pages are delivered using the HTTP protocol. By default 
the NetEchoes™ instrumentation script and collected data are sent using HTTP. However, 
frequently data is deemed sensitive and transmission using a Secure Sockets Layer (SSL) is 
10 commonly used. SSL transmission sends only encryption data across the internet. It the 
primary method used to deliver sensitive information such as credit card numbers and user 
passwords across the internet. In the case of SSL delivered pages, the NetEchoes™ tag 
embedded on the page must reference the instrumentation script using SSL and the script 
must deliver the data using SSL. To simplify this process a "smart" NetEchoes™ Tag has 
15 been developed which is suitable for use on both secure (SSL-delivered) pages and unsecure 
pages. The general pattern of the tag is illustrated below: 

< SCRIPT LANGUAGE = " JavaScriptl . 2 " > 
var kl_siteID = "PUT_SITE_ID_HERE" ; 
var kl_tagProtocol = " " ; 
if (location. protocol == "https:") 

kl_tagProtocol = " s " ; 
var kl_tag = "<SCR M + "IPT LANGUAGE= ■ JavaScriptl . 2 ■ " + 
"SRC=http" + kl_tagProtocol + 

'•: //stats. klsoft.com/script/Site_" + kl_siteID + kl_tagProtocol 
+ "_ePulse. js M + 

"></SCR" + n IPT>"; 
document . write (kl_tag) ; 
</SCRIPT> 

This smart tag detects the protocol of the page upon which it is embedded: either 
O secure (https) or unsecure (http) and adjusts the URL and script delivered appropriately. 
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Instrumentation of the Document 

Instrumentation requires that the JavaScript "hook into" the event model for the 
document. This is challenging, because the document may already have JavaScript event 
handlers in place. The instrumentation script must be able to dynamically insert JavaScript 
handlers for the necessary events that it wishes to track without disrupting existing handlers. 
40 This is one of the most technically challenging aspects of instrumentation. Described below is 
a technique that allows instrumentation without disruption referred to as "Dynamic 
Augmentation of Existing Event Handlers." 
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Dynamic Augmentation of Existing Event Handlers 

In the following, we use the "onload" event of a document to illustrate to concept of 
Dynamic Augmentation of Existing Event Handlers, but the concept applies to dynamically 
augmenting the handler for any event. 

The event handlers of a page can be set up by the producer in one of two patterns. The 
first is as a attribute of an HTML tag, the second is as the setting for a JavaScript property. 
These are illustrated in FIGS. 65A and 65B. 

Event handlers are commonly specified in commercial web pages using both of these 
methods. In dynamically augmenting these handlers, NetEchoes™ must replace the existing 
handler (in this case onLoadHandler()) with a new event handling method that both preserves 
existing behavior and adds the necessary NetEchoes™ behavior. 

Assume for the moment that NetEchoes™ behavior is embodied in the JavaScript 
method netEchoesOnLoadHandler(). Then, in the augmented case, the onLoad event must 
execute both the existing onLoadHandler() and the new netEchoesOnLoadHandler(). 

The Dynamic augmentation arranges for this to occur by dynamically creating a third, 
anonymous function according to the following pattern: 

function anonymous () 

{ 

onLoadHandler ( ) ; 
netEchoesOnLoadHanlder () ; 

} 

This new function is replaces the original handler (onLoadHandler()) as the new 
handler by setting the appropriate JavaScript property: 

Window. onload = anonymous ; 

This example is a simplification of the dynamic augmentation process. Other aspects 
that must be addressed include: 
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• The effects of the change in scope of the original handler: onLoadHandler(). 
Mitigation of these effects may include addition of parameters and substitution 
of the "this" pointer with an artificially manufactured "kl_this" pointer. 

• Arguments (if any) of onLoadHandler() 

• The return value of onLoadHandler(), if one exists, must be captured and 
returned by the anonymous function. 

• Differences in event handling models of Netscape and Microsoft browsers. 

The JavaScript source for the complete handler augmentation is listed below. 

// Install an event handler, If one already exists, replace it with 
//an anonymous function that calls both the original code and the 
// iEchoes hander. 

function kl_setHandler (kl_method, kl_newFunc, kl_oldFunc) // nf is newFunc, of is oldFunc 

{ 

// look to see if there is any existing handler in place 
if (kl_oldFunc) 
{ 

var kl_ofuncBody = kl_oldFunc . toString ( ) ; 
// a stands for arg 

// The regular expression pulls out the argument list of the old 

// function, for example if the function string is "f{a, b, c) {var i ...}", 

// then the match rule puis out "a, b, c" in a[l] 

var a = kl_ofuncBody .match (new RegExp ("\\ ( ( T\\> 1 *) \\) ") ) ; 



// if there were no arguments, manufacture as single argument e, which will 

// become the event argument. The event argument is used by Keylime's code 

// only when the browser is Netscape, but it doesn't hurt to create the argument 

// in all cases. 

a = (a[l]. length >0 ? a [1] : "e") ; 

// replace the arguments in the new event hanlder to match 
// the original event handler the user set in place, 
if (kl_isNav) 

// The reg expression below matches the argument list and the replace 
// replaces it with the new arg list. 

kl_method = kl_method. replace (new RegExp ( "\\ ( 1 *\\) " , "g" ) » " < r, +a+" ) " ) ; 

if (kl_ofuncBody . indexOf (kl_method) < 0) 

{ 

// b is bracket character index 

var b = kl_ofuncBody . indexOf ( " { " ) ; 

if ( (kl_isNav || kl_of uncBody . indexOf ( "anonymous" ) > 0) && b > 0) 

kl_ofuncBody = kl_of uncBody . substring (b, kl_of uncBody . length) ; 

else if ( kl_of uncBody . indexOf ( "function" ) >= 0 && b > 0) 

kl_of uncBody = k l_of uncBody . substring (kl_of uncBody. indexOf 
("function") + 9, b) ; 

// escape any newline chars in the old function body, 
// to exscaped newlines 

kl_ofuncBody = kl_ofuncBody. replace (new RegExp ( "\\n" , "g" ) , "\\n") ; 

// replace any occurences of the this pointer to kl_this. The newly defined 

// user function will be passed the original this pointer as kl_this. 
// This is 

// necessary since the scope of the original code is being replaced. 
kl_ofuncBody = 
kl_of uncBody. replace ( 
new RegExp 

(" ( ["a-zA-Z0-9$ ] )this( ["a-zA-Z0-9$_] ) ", "g") , "$lkll_this$2" ) ; 
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//kl_ofuncBody = kl_of uncBody . replace (/\n/g, 1f ; ") ; 
// Escape all ' characters 

kl_ofuncBody = kl_of uncBody . replace (new RegExp ( " 1 " , "g" ) , "W 1 "); 
// escape all " characters 

kl_ofuncBody = kl_of uncBody . replace (new RegExp ('"■ , "g" ) , ' W 1 '); 

// create then function body for the new event handler, 
var kl_newFuncBody = 
«{" + 

"var aO = ' " + a + " • ; " + 
"var of b = ' " + kl_of uncBody + " ' ; " + 
"var f = new Function ( aO, ' kll_this', of b) ; " + 
"var rv = f ( » + a + ",this),-" + 
kl_method + " ; " + 

"return rv; " + 

" } " ; 



return new Function (a, kl_newFuncBody) ; 

> 

else 

return kl_oldFunc; 

) 

return kl_newFunc; 

} 



Interaction between Instrumentation Script and DOM 

FIG. 66 illustrates the objects involved in the instrumentation script and their 
interactions. Given that HTML pages for e-business sites are typically constructed 
dynamically via CGI execution or other server-side page generation technique, the 
NetEchoes™ tag will in most cases have to be integrated into the web site's server setup and 
web page construction process. The most straight forward solution to inserting the 
instrumentation into each page seems to be server side include statement, server module side 
includes are supported by nearly all commercial web servers. They allow a reference to a 
document to be embedded in other documents. When a document is served to a client the 
reference is resolved and the additional content written on the stream. This model avoids a 
secondary request from the client to fetch the contents of the reference. 



In a preferred implementation NetEchoes is designed to accommodate users of 
phone modems with speeds as low as 28.8 kbps. Accordingly, the size of profiles 
downloaded to clients from the server module need to selected appropriately. In addition, the 
HTTP protocol is "connectionless", and thus requires that a new connection to the remote 
HTTPD server be made for each request. A request for a user profile therefore requires a 
new connection across the internet and the download of a potentially large document. Note 
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5 that the instrumentation of the document takes place immediately after the document is 
loaded. From the user's perspective, any additional data transfer at this point would have a 
noticeable effect on the response time of the browser. In light of these issues, the 
NetEchoes™ product either collects all features and organizes them through post processing 
or performs a variation on profiling by using a custom instrumentation script that instruments 
1 0 only the portions of the document that are of interest. 

Instrumentation of Applets, Plug-ins and ActiveX controls 
15 The instrumentation of applets, plug-ins and ActiveX Controls is not as 

straightforward. For Netscape browsers, applets and plug-ins are accessed via Netscape's 
LiveConnect technology. The plug-ins must be wrapped by a Java API. For Microsoft's 
Internet Explorer, applets and controls are accessed via ActiveX technology. All ActiveX 
controls can be accessed directly from JavaScript without modification. 



array properties which contain references to applets, embedded objects (ActiveX controls) 
and in the case of Netscape, plugins. 



Feature Tracking 

Within HTML pages, features (UI controls) can only appear within the context of 
HTML forms. Tracking features then, is the same tracking controls on forms. In general, the 



30 controls generate JavaScript events when they are used. This makes is especially easy for 
NetEchoes™ to track the details of what goes on in forms. Table XXXXXVII presents the 
possible form controls along with the properties and events available from within JavaScript. 




Regardless of the JavaScript implementation, embedded objects such as ActiveX 
controls and applets are accessible via the document object arrays. The document object has 
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Table XXXXXVII 
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As Table XXXXXVII illustrates, most controls have a value and name, plus events such as 
onclickQ or onchangeQ that make the use of them inherently trackable. 



Standard Data Collection 

Table XXXXXVIII below lists the client-side data that is collected by the standard 
instrumentation script. 

Table XXXXXVIII 

NetEchoes™ Records and Field Descriptions 



Yype Field Name Qualifier Description 



TSTAMP 


First in log 
entry 


The first record of a log entry giving the start 
time from to which all other records for a page 
will be relative. The event time for each 
subsequent log record will be the start time in 
this record plus a delta time in the event record. 


start time 




The start time for the entry. This is in GMT 
time. Note there is a possibility of inaccuracy if 
the client's clock is set incorrectly. 


time zone 




+/- minutes from GMT. Time zone of the client 
as reported by the client. 
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company 
identifier 




company identifier for the site 




site identifier 




site identifier for the page. 








session identifier for the site domain set for 




session 
identifier 




duration of the browser session or time interval 






(30 min). Different sites will have different 
reported session identifiers. The start of a 
session is indicated by a + appended to the id. 


PAGE 




Second in log 
entry 


Description of the page being viewed. This 
record always follows immediately the time 
stamp record and does not have a delta time. Its 
fields are relevant to all subsequent event 
records within the log entry. 




location 




document URL giving the location of the file for 
the page 




parent window 


Window 
INFRAME 


if present, gives the document URL that holds 
the frame for this document 




document href 


HREF != 
Location 


if present, the URL that was used initially to 
retrieve this document. This would include the 






query for example. 




referrer 


via LINK 


if present, the URL that loaded this document 




page view num 




A sequence number for each page view. Starts at 
0 for the first page view of the session and 
increments by 1 for each successive page view. 
Restarts at 0 for each session. 


LOAD 




LOAD Event 


After timestamp & page records, typically the 
first event record indicating the start of viewing 
for the page. Exceptions are scripting errors, 
image aborts and click throughs that occur 
before the loading has completed. 




delta time 




the delta time in milliseconds from the start time 
for the record. 




load time 




time in milliseconds to load the page 




last loaded 


NAV ONLY 


if present, the last loaded image during page load 


GOTO 




LINK Event 
after LOAD 


generated whenever a link is pressed within the 
document. If this record is not present, then 
another means was used to go to next document. 
Click throughs on the page before the document 
loaded are not recorded. 




delta time 




the delta time in milliseconds from the start time 
for the record. 




link 




goto link giving the next document to be loaded 




advertisement 


IE ONLY 


image used to activate link assumed to be an ad. 


FORM 




Focus Time > 0 


generated whenever forms are present in the 
document. This is generated only when the focus 
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time is greater than zero. 


Hpltfl timp 

lid Id LllliW 


the delta time in milliseconds from the start time 
for the record. 


reason = 
SUBMIT, 
RESET, 
ABANDON[i] 


reason the record was generated for the form. 
For abandoned forms the form index number is 
given and the abandon form report is generated 
only if the focus time is greater than 0. 


name Has Name 


if present, the name for the form 


number 
elements 


number of elements in the form 


number 
elements set 


number of elements the user set in the form 
before this record was generated 


last element 


last element entered before form was abandoned. 


focus time 


accumulated time in millisecs spent by user in 
the form 



Site-specific 
form info fields 
0 thru 9. 



Optional. Only 
delivered when 
reason is 
SUBMIT and 
this information 
is tracked by 
the form code. 



submission. These fields contain string values 
that are the values of input fields on the form. 
The values are interpreted by the DataMining 
servers, usually for the purpose of page-view 
filters. These values are set only for certain 
forms. They can be set either by a special site- 
specific instrumentation script or by end-user 
code the modifies the kl_InfoAry[n] property on 
the submitted form. 



UNLOAD 


UNLOAD 
Event 


last record in entry. 


delta time 




the delta time from document load that the 
document was viewed. 


active time 


NA 


accumulated time the document had focus 


aborted images 




If present, the number of aborted images at time 
of document unload. This would occur if the 
user licked through the page before it completely 
loaded . 


page specific 
info 




Page specific info. Usually meaningful only 
when taken in context with the URL for the 
page. Most common use is for "bag of tags" 
whereby the client sets the value 
"window.epulse_info" on an instrumented page. 
That value is picked up by the script and placed 
in this field. Occasionally, a client-side 
javascript that has been customized for a client 
will use this field as well. 


ABORT 


First 5 images 


generated whenever an image has been aborted 
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such as the user pressing the stop button or 
changing a page before loading has completed.. 




delta time 




the delta time in milliseconds from the start time 
for the record. 




image 




href for the image aborted image 




link 


IE Only 


if present, the link associated with the aborted 
image 








generated whenever a error within the document 


ERROR 






(image loading and scripting errors) has 
occurred. In some cases, scripting errors cause 
the script to be abandoned, and prevent the page 
from being instrumented. 




rif*ltfi timp 




the delta time in milliseconds from the start time 
for the record. 




reason 




the reason the error occurred in free text 








the URL of the document that generated the 




document.href 


JScript error 


error if different from current document. For 
image errors this is the image source. 




line number 


JScript error 


the line number where the error occurred. 








generated when session id is first created to 








summarize session environment. Should be used 


SYSTEM 






with other header data included in request or 
post. Will be always the first event record 
reported from a session. 




delta time 




the delta time in milliseconds from the start time 
for the record. 




screen 




screen dimensions and color depth 
(<width>x<height>x<pixel-depth>). 




window 




window dimensions (<width>x<height>). 




plug-ins 


Nav Only 


list of client plug-ins and helpers giving mime- 
types and plug-ins for each mime-type. 
(NAVIGATOR ONLY). 



From these collected measures, all the questions first proposed can be answered: 

• How long did the page take to download (from the client's perspective)? 

• What was the previous page that led the user to this page? 

• How was the user fed to the site? Was it fed by a search engine, link from within 
an email, advertisement on another site, link from another site, or bookmark or 
type in? 

• How did each visitor behave on the site? Did they make purchases, sign up for 
newsletters, or behave in other desirable ways? 

• How did visitors fed from different sources behave differently? Which referral 
source fed the most visitors who purchased, subscribed or behaved in other 
desirable ways. 
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5 • What is the visitor breakdown for the site. How many visitors are returning or 

new? Of those returning, how many have behaved in desirable ways in the past 
(i.e. have they purchased, subscribed, or taken another action)? 

• What advertisements did the user actually view (ad impressions)? 

• Which elements (images and ads) on the page were the slowest to load? 
10 • What were the most commonly clicked on links of page? 

• How may pages on the site were viewed previously to this one? 

• How long was the user active on the page? 

• What advertisements were clicked-thru on the page? 

• Did the user hesitate during any part of a process or task? 
15 • Was any operation, task or form abandoned? 

• When a form was abandoned, what was the last field filled in by the user? 

• How the user left the site (advertisement selection, other URL, typed text in the 
GOTO box, etc.)? 

• Where did the user go when they left the site? 

20 • How long did each page take to load from the user's perspective? 

• How many of the pages the user viewed came from cache vs. a load from the site? 
Q • What path did the user actually take through the site (modified to include cached 
jO pages)? 

^ • Did the user interrupt the loading of the page by pressing the stop button, clicking 

^ 25 on a link, etc.? 

^ • Did the user leave the site immediately after interrupting the download of a page 

j fj (interrupt abandons)? 

• What browser and platform (operating system) was the user using? 
s • Did any errors occur while the user was interacting with the page? 
y 30 • Did the user make any purchases while on the site? 

f'^z 

ru 

y Data Transmission Methods 

u 35 

Transmitting the data collected by the instrumentation script back to a remote server 
elsewhere on the internet poses several problems. First, in order to pass through the intervening 
appliances on the net such as firewalls and routers, the connection must be made on an open port. 
The solution to this problem is to use port 80, the HTTP protocol port. It is on this port that the user 
40 fetched the instrumented page in the first place. It is therefore the logical means to send the collected 
data back. 

FIG. 67 illustrates the basic problem of data transmission from a client browser to a remote 
server on the internet. Sending the data requires some creativity. Generally speaking, JavaScript has 

134 




5 no network privileges. It cannot simply open a socket connection to the data host on port 80 and 
transmit the data using the PUT operation of the HTTP protocol. Three indirect means of data 
transmission have been examined: 

• Data transmittal using 1x1 pixel image request 

• Data transmittal with cookies 

1 0 • Data transmittal using hidden form fields 

These three alternatives are described below. The most promising of these is the data 

transmittal by lxl pixel image request. This method is currently in use for NetEchoes™ 
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Data Transmittal by lxl pixel image request 

Generally speaking, a JavaScript operating within an HTML page has very little 
networking privileges. It cannot arbitrarily send data across the network. In fact, the only 
networking capabilities available to JavaScript is the ability to request an image from a 
remote server. A JavaScript fragment illustrating this capability is shown below: 

var picture = new Image (); // constructs an image object 



// the following gets the Keylime logo from the remote 
25 // Keylime web server 

picture. src = http : //www. keylimesof tware . com/logo . gif " ; 

The above fragment makes network request of the http server at 
fU 30 www.kevlimesoftware.com to obtain the image file logo.gif. 



Interestingly enough, an inherent capability of URL requests is that they can be 
accompanied by a parameter specification also known as a "query string " within the URL. 
The query string is set off in the URL by the use of a '?' character and is of unlimited length. 
35 (The details of a URL format can be found in the W3C committee HTTP 1.1 protocol 
specification.) Therefore, the JavaScript request for an image file can be accompanied by an 
arbitrary data payload disguised as a query string. 

Using this technique, the previous JavaScript fragment used to request an image can be 
40 adjusted to deliver a data payload. 
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var picture = new Image (); // constructs an image object 



var payload = "Any payload string you wish to send" ; 
payload = escape (payload) ; // encode payload string to abide 

// by http conventions. 
// The following requests a lxl pixel image, but also 
// encodes as part of the request, an arbitrary data 
// payload. 

// Collection.gif is a lxl pixel image served up by 
// the stats.klsoft.com web server 
picture. src = 

http : //stats . klsof t . com/Collection . gif " + u ?" + payload; 



The web server at stats.klsoft.com can be adjusted in a number of ways to record the 
data payload sent along with the image request. The easiest way is to configure the web server 
to create logfiles containing the complete text for every URL requested. In this way, the 
payload data sent as part of the URL will appear in the server log files. 

Data Transmittal with Cookies 

Cookies are often used to give the illusion of persistence over the connectionless and stateless 
HTTP protocol. They were originally created to support CGI programming and are implemented as 
an extension to the HTTP protocol . The data in cookies is automatically transmitted between the 
browser and the HTTP server without the knowledge of the user. 

A cookie consists of a named value, plus four optional properties which control the lifetime, 
visibility and security of the cookie. The lifetime parameters are defined below: 

Expires - Specifies the lifetime of the cookie. By default cookies last for the duration of a 
web browser session. Setting the Expires property to a future data allows the cookie to be 
persisted until that time. 

Path & Domain - Control the visibility of the cookie. Cookies are automatically transmitted 
to the servers for which they are deemed visible at the time a connection is made. 

Secure - This property specifies how cookies are transmitted over the network. If this 
property is set the cookie is transmitted only the client connection to the server is secure. 

The cookies associated with a particular page are made available to JavaScript through the 
cookie property of the document object. 
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Using cookies as a means to transmit data to the server requires that the data be sent in chunks 
with a size of 4k bytes or less. The maximum size of a cookie, including the NAME of the cookie, is 
4k. Note also that the visibility of the cookie will have to be adjusted such that the value of the 
cookie is transmitted when a request is made for any document at the source site. 

When the user leaves the site where the instrumented page was encountered, the last block of 
data must have the Expires property set in order to persist the value of the cookie on the client until 
the next time the user connects to the source site. 

To avoid overflowing the 4kb limit on cookie size, the instrumentation script could take 
advantage of compression as well as the use of multiple cookies. The maximum number of cookies 
that a browser is required to manage is 300, more than enough for the purposes of NetEchoes™. 

Data Transmittal with Hidden FORM Fields 

Transmittal of monitoring data can also be performed using what is commonly 
referred to as a hidden FORM. Elements of a FORM defined with <INPUT TYPE-hidden> 
are not visible to the user. A FORM containing a single hidden element can be used to 
transmit data to a server without altering the appearance of the HTML page in which it is 
embedded. 

A simple example of a hidden FORM is shown below: 

<FORM NAME=myform ACTION="/cgi-bin/inputdata.pr METHOD=GET> 

<INPUT TYPE=hidden NAME="mydata"> 

</FORM> 

A script may refer to the data item as myform.mydata and set the value to a string 
containing a compressed stream of monitoring data. 

The METHOD tag in the form definition controls whether the data is sent to the server 
appended to the ACTION URL (GET) or sent in the body of the HTTP request (POST). 
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Examples are provided blow expressed in HTTP protocol notation. 

GET http://wwwJimesoftxom/cgi-bin/inputdata.pl?mydata=<data> HTTP/1 . 1 

POST http://www.limesoft.coni/cgi-bin/inputdata.pl HTTP/ 1.1 

.... [additional header entries here] 

mydata=<data> 

The hidden FORM approach to data transmission has an advantage over the cookie 
method in that the instrumenting script has control over when the data is sent. The data may 
be sent to the server at any time by invoking the submitO method of the FORM. 

Controlling when data is sent is possibly more important than how. The script 
embedded in the instrumented page must strive to send monitoring data to the server at a time 
when the user is not initiating any other server requests otherwise the user has the impression 
of degraded performance. Ideally the data should be sent during periods of inactivity such as 
when the user pauses after a document has loaded to view text or images. 

Customizing Data Collection 

In addition to the standard metrics collected, the instrumentation script can be 
customized to collect any additional information (visible to JavaScript) for the page view. 

Bag Of Tags 

In addition to customization of the JavaScript, the basic script itself provides a way for 
producers to adjust the information delivered by interacting with the script on the page. 
Simply by setting a property, the client can deliver "free-format" data to the NetEchoes™ 
server. More specifically, the producer can embed JavaScript like that shown below onto the 
page: 

Window. epulse_info= Window. epulse_info + purchasePrice . toString () + 
qtyOrdered . toString ( ) ; 

Statements such as these, give the "epulse_info" property values that are transmitted 
back to the NetEchoes™ server module. The NetEchoes™ server module can then take 
special, site-appropriate action with this data. 
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Segmenting Users Based on Behavior 

Because the session model for contains all the details of the users segment, the 
DataMining server can include custom logic for each site that scans user sessions to identify 
desirable user behavior. Those quality behaviors can then be permanently associated with the 
user in question. In this way, when the user returns for successive sessions, the user's history 
(in terms of his quality behavior) can be queried from the DB. There is a variety of use for 
such information, including incorporation into personalization systems, reports, etc. 

Complete NetEchoes™ Instrumentation Script 

Appendix A includes the complete NetEchoes™ instrumention script. Although 
potentially customized to collect additional data, all other scripts are derived from the basic 
pattern set forth in Appendix A. 

NetEchoes™ Server 

The NetEchoes™ server components are independent from both the Producer Web 
Site and the Browser. They are typically housed in a separate data operations center. The data 
operations center houses computer hardware that provides the NetEchoes™ server 
capabilities. NetEchoes™ server components provide a wide variety of capabilities, including 
serving up the instrumentation script, receiving the collected data, publishing reports showing 
real-time user activity, etc. However, a significant aspect of the NetEchoes™ server is real- 
time modeling of user sessions: 

Real-Time Modeling of User Sessions 

As collected data is received from the Instrumentation Script, it is immediately loaded into the 
memory of the DataMining server (a NetEchoes™ server sub-component). The purpose of the 
DataMining server is to take collected page view data and construct an in-memory model of 
each in progress user-session on the instrumented site. The in-memory model includes all of 
the relevant information about the session including: 



• When the session started 

• The user of the session (based on cookies) 



139 



• Where was the user feed from, what external site, what email, ad, etc. 

• The pages that have been viewed in the session (in sequence) 

• The user's experience on each page; e.g. Did he submit a form, press certain 
links, view certain ads, etc. 

• Whether or not this user is a new or returning user. If returning, what is the 
user's history. 

• Information about how the session ends. Did the user click on and external link 
or ad, or leave the site in some other way. 

The information for all active (in-progress) user sessions is kept in-memory for as 
along as the session is active. However, as the session state changes (new information pours 
in from the client) the session state is also persisted to a relational data store. In this way, all 
the information modeled in-memory is also reflected in relational tables. Just as the in- 
memory session information is updated in real-time, so is the relational database. This is done 
so that all reporting functionality of the NetEchoes™ server can run reports of the relational 
database and still reflect real-time producer site statistics. 

Handling the sessions in-memory by active objects enables a variety of capabilities: 

• The information can be used to give real-time analysis of how the sited is 
being used 

• The information can be taken advantage of by personalization engines to 
dynamically modify and target the content transmitted. 

Although the above application has been described primarily in the context of a 
system in which client computers are linked to a server computer through a local area network 
or the internet, one skilled in the art can readily appreciate that the present invention is equally 
applicable to the monitoring of usage of other types of electronic devices (e.g., celluar phones, 
personal digital assistants) in wireless and other environments. Thus the application is meant 
only to be limited by the scope of the appended claims. 
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